高级检索
    韩冰, 张鑫云, 任爽. 基于三维点云的卷积运算综述[J]. 计算机研究与发展, 2023, 60(4): 873-902. DOI: 10.7544/issn1000-1239.202220135
    引用本文: 韩冰, 张鑫云, 任爽. 基于三维点云的卷积运算综述[J]. 计算机研究与发展, 2023, 60(4): 873-902. DOI: 10.7544/issn1000-1239.202220135
    Han Bing, Zhang Xinyun, Ren Shuang. Survey of Convolution Operations Based on 3D Point Clouds[J]. Journal of Computer Research and Development, 2023, 60(4): 873-902. DOI: 10.7544/issn1000-1239.202220135
    Citation: Han Bing, Zhang Xinyun, Ren Shuang. Survey of Convolution Operations Based on 3D Point Clouds[J]. Journal of Computer Research and Development, 2023, 60(4): 873-902. DOI: 10.7544/issn1000-1239.202220135

    基于三维点云的卷积运算综述

    Survey of Convolution Operations Based on 3D Point Clouds

    • 摘要: 随着深度相机、激光雷达等3D扫描设备的普及,用点云表示3D数据的方法越来越流行,对点云数据的分析与处理也引起了视觉研究领域的极大兴趣. 一般来说,由于分布在3D几何空间中的点云是无序的,可以形成特定的结构,因此学习到的特征表示应该具备排列不变性、旋转和平移不变性、形状区分性. 近年来,越来越多的研究人员基于点云的这些特性采用深度学习这一人工智能领域中的主流技术来处理分析点云数据. 其中,卷积神经网络使用的卷积运算具备权重共享、局部聚合和变换不变等优点,减少了训练参数的个数,并具有较强的鲁棒性,可以有效地降低网络复杂度并提升网络性能,因此在各种2D视觉问题(如图像、视频)上的研究及应用已经相对成熟,这也引起了研究人员的高度关注并尝试将其引入到点云处理任务中. 但传统的标准卷积运算往往无法直接作用于点云这种不规则数据上,一些研究人员进而对卷积运算及其卷积算子展开了深入的探索,并提出了多种卷积策略和网络以提高计算效率和算法性能. 为了促进之后的研究,首先对现有点云研究中所使用的卷积方法进行了概述,包括基于投影的方法、基于体素的方法、基于晶格的方法、基于图的方法和基于点的方法. 之后着重针对直接处理3D点云的卷积算子和网络的最新进展进行了全面的综述,主要分为离散卷积和连续卷积,此外还对使用不同卷积算子的网络在处理点云的分类和分割等任务上的性能进行了全面地分析与对比. 最后针对现存的问题与面临的挑战进行了进一步分析,并探索了未来可能的研究方向,希望为点云未来更深入的研究提供新思路.

       

      Abstract: With the popularity of three-dimensional (3D) scanning devices, like the depth cameras and LiDARs, using point clouds to represent 3D data becomes ubiquitous. Compared with two-dimensional (2D) images, point clouds can provide richer information and capture more 3D structures. Therefore, point cloud learning has recently attracted a surge of research interests in computer vision community and promoted various emerging applications, such as robotic manipulation, autonomous driving and augmented reality. Generally, the learned representations of point clouds should have the characteristics of permutation invariant, transformation invariant (e.g., rotation and translation) and shape distinguishability. Therefore, in recent years, more and more researchers have carried out research on using deep learning (DL) to deal with point clouds. Among them, the convolution operations in convolutional neural networks (CNNs) have the characteristics of weight sharing, local aggregation and transformation invariance, which can effectively reduce the complexity of the networks and the number of training parameters. Meanwhile, CNNs have been successfully used to solve various 2D vision problems of images and videos with strong robustness. Therefore, CNNs attract great attention of researchers and are introduced into some point cloud tasks. However, the traditional standard convolution operations cannot directly act on the irregular data such as point clouds. Therefore, some researchers carry out in-depth explorations on the convolution operations and then propose a variety of convolutional strategies and networks to improve the computational efficiency and algorithm performance. To stimulate future research, we first summarize convolutional methods used in existing point cloud research, including projection-based methods, voxel-based methods, lattice-based methods, graph-based methods and point-based methods. After that, we focus on the recent progress in convolution operators and networks based on point clouds mainly including discrete convolutions and continuous convolutions. In addition, the performances of networks using various point-based convolution operators in some related tasks (such as classification and segmentation) are comprehensively analyzed. Then we quantitatively compare these methods on some synthetic datasets and real-scanned datasets, and obtain relative state-of-the-art (SOTA) methods of each point cloud task. Extensive experiments can verify the performances as well as the effectiveness of these proposed methods. Finally, aiming at some existing problems and challenges, we also present insightful observations together with inspiring future research directions.

       

    /

    返回文章
    返回