Abstract:
Large Language Models (LLMs) have attracted significant attention for their exceptional reasoning and long-context capabilities. However, with the growth in model parameter scales and increasingly complex application scenarios, LLM inference faces severe challenges regarding resource utilization and efficiency. On one hand, the massive parameter scales and the nature of autoregressive generation necessitate the optimization of computation, memory, and communication resource allocation to maximize hardware resource utilization. On the other hand, real-world online serving scenarios introduce highly dynamic concurrent requests and variable computational loads, making it difficult to balance the dual service quality requirements of high throughput and low latency. This paper provides a systematic survey of research progress in LLM inference optimization, analyzing the design principles and efficacy of existing techniques across three dimensions: Model, Algorithm, and Service. The Model layer significantly reduces resource overhead while maintaining model precision through quantization, sparsification, and model architecture optimization. The Algorithm layer focuses on reconstructing computational logic, breaking performance bottlenecks associated with memory bandwidth and serial generation by optimizing data flow and operator execution patterns. The Service layer concentrates on system-level resource orchestration, achieving Pareto-optimal trade-offs between throughput and latency under specific workload conditions through dynamic request scheduling, architectural disaggregation, and heterogeneous collaboration. Furthermore, we examine the limitations and applicability boundaries of existing techniques, analyze the synergies and conflicts among different optimization paths, and finally discuss future research directions for LLM inference optimization.