ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2017, Vol. 54 ›› Issue (6): 1316-1325.doi: 10.7544/issn1000-1239.2017.20170095

Special Issue: 2017计算机体系结构前言技术(一)专题

Previous Articles     Next Articles

Real-Time Panoramic Video Stitching Based on GPU Acceleration Using Local ORB Feature Extraction

Du Chengyao1, Yuan Jingling1,2, Chen Mincheng1, Li Tao3   

  1. 1(School of Computer Science and Technology, Wuhan University of Technology, Wuhan 430070); 2(Hubei Key Laboratory of Transportation Internet of Things (Wuhan University of Technology), Wuhan 430070); 3(Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL, USA 32611)
  • Online:2017-06-01

Abstract: Panoramic video is a sort of video recorded at the same point of view to record the full scene. The collecting devices of panoramic video are getting widespread attention with the development of VR and live-broadcasting video technology. Nevertheless, CPU and GPU are required to possess strong processing abilities to make panoramic video. The traditional panoramic products depend on large equipment or post processing, which results in high power consumption, low stability, unsatisfying performance in real time and negative advantages to the information security. This paper proposes a L-ORB feature detection algorithm. The algorithm optimizes the feature detection regions of the video images and simplifies the support of the ORB algorithm in scale and rotation invariance. Then the features points are matched by the multi-probe LSH algorithm and the progressive sample consensus (PROSAC) is used to eliminate the false matches. Finally, we get the mapping relation of image mosaic and use the multi-band fusion algorithm to eliminate the gap between the video. In addition, we use the Nvidia Jetson TX1 heterogeneous embedded system that integrates ARM A57 CPU and Maxwell GPU, leveraging its Teraflops floating point computing power and built-in video capture, storage, and wireless transmission modules to achieve multi-camera video information real-time panoramic splicing system, the effective use of GPU instructions block, thread, flow parallel strategy to speed up the image stitching algorithm. The experimental results show that the algorithm mentioned can improve the performance in the stages of feature extraction of images stitching and matching, the running speed of which is 11 times than that of the traditional ORB algorithm and 639 times than that of the traditional SIFT algorithm. The performance of the system accomplished in the article is 59 times than that of the former embedded one, while the power dissipation is reduced to 10W.

Key words: panoramic video, image stitching, heterogeneous computing, embedded GPU, oriented FAST and rotated BRIEF (ORB)

CLC Number: