Latency Optimization in Real-Time Multimedia Transport: Architecture, Progress and the Future

Real-time multimedia transmission is one of the most important applications of the Internet, with the applications such as videoconferencing, cloud gaming, virtual reality and so on. In the same time, the real-time multimedia transmission systems therefore demand high requirements for the end-to-end transmission latency. Among them, latency fluctuation is the most challenging problem in latency optimization. However, traditional ‘best-effort’ transmission services in the Internet cannot meet the requirements of latency fluctuations for real-time multimedia transmission in many cases. This paper first elaborates on the main challenges faced by real-time multimedia transmission. Secondly, it analyzes the key issues that need to be addressed to optimize the latency of real-time multimedia transmission. Based on these issues, two key paths (control path and data path) and five core components in the architecture of real-time multimedia transmission system are summarized. Around the technologies involved in each component, representative research results are summarized and discussed, especially for the research efforts in the recent years. Based on the analysis above, the research branches for real-time multimedia transmission and low-latency applications are summarized, and the optimization algorithms and applications for each research branch are reviewed. A key finding in this paper is that the fluctuation of latency is the key metric that research needs to work on. Finally, this paper also discusses and proposes possible future research directions for the readers.