ISSN 1000-1239 CN 11-1777/TP

计算机研究与发展 ›› 2018, Vol. 55 ›› Issue (12): 2764-2774.doi: 10.7544/issn1000-1239.2018.20170891

• 图形图像 • 上一篇    下一篇



  1. 1(西南科技大学计算机科学与技术学院 四川绵阳 621010);2(西南科技大学四川省军民融合研究院 四川绵阳 621010);3(达闼科技成都有限公司 成都 610100);4(法国国家科学研究中心电子、计算与成像科学实验室(勃艮第大学) 法国第戎 21078) (
  • 出版日期: 2018-12-01
  • 基金资助: 
    国家自然科学基金项目(61872304,61802320,61872066, 61502083);四川省科技基础条件平台项目(2017TJPT0020);四川省科技计划项目(2017KZ0023);四川省军民融合研究院开放基金项目(JMRH01)

Design and Research of Data Glove Based on Visual Information Fusion

Wang Fupan1, Wu Yadong1,2, Yang Wenchao3, Yang Fan4, Hou Jiaxin1, Liao Jing1   

  1. 1(School of Computer Science and Technology, Southwest University of Science and Technology, Mianyang, Sichuan 621010);2(Sichuan Civil-military Integration Institute, Southwest University of Science and Technology, Mianyang, Sichuan 621010);3(CloudMinds Chengdu Co., Ltd, Chengdu 610100);4(Laboratory of Electronic, Computing and Imaging Sciences, French National Centre for Scientific Research (LE2I-UMR CNRS 6306) (University of Burgundy), Dijon, France 21078)
  • Online: 2018-12-01

摘要: 基于数据手套的手势交互是人机交互技术的重要实现方式,手势识别和空间定位效率对虚拟交互任务中的沉浸感和体验感有重要影响.目前主要的数据手套产品的复杂结构导致相应建模及手势识别算法设计复杂度高,难以兼顾成本和效率.面向应用需求,完成了一种融合视觉信息的数据手套应用系统方案设计.首先,在手势识别上,通过将3指弯曲度数据和视觉采集模型相结合的方法,有效简化了手势的建模和匹配过程,实现对特定手势的快速识别.其次,在空间位置定位上,提出了一种高效的基于特征校验的霍夫变换算法:通过摄像头捕获发光球的圆形轮廓以计算其三维空间位置;使用基于直径方程的改进型霍夫梯度算法降低圆形检测识别复杂度;对于连续帧图像,采用了基于可信度的特征校验法则进行快速处理,进一步提高算法整体效率.最后,该方案在沉浸式中学虚拟实验系统中应用,系统运行稳定,具有良好的交互体验.

关键词: 数据手套, 手势识别, 视觉融合, 人机交互, 特征校验

Abstract: Gesture interaction based on data glove is an important way to realize human computer interaction . The effect of gesture recognition and spatial positioning has an important influence on the immersion and experience of virtual environment interaction tasks. The complex structure of typical products lead to high complexity of modeling and gesture recognition algorithm,and it’s hard to balance the cost and efficiency.Aiming at the requirements of application,we come up with a design of data glove system which combines the sensor data with visual information in this paper. A rapid mapping that convert the finger’s curvature value to virtual hand joints position is realized by a hierarchical template matching method,which simplifies the process of gesture’s modeling and matching effectively. In order to obtain the spatial information for gesture interaction, an efficient feature-checking Hough algorithm is proposed to locate the data glove. We use the camera to capture the circular contour of the luminous ball on the glove and calculate it’s position in three-dimensional space, and use a modified Hough-Circle-Detection algorithm based on diameter equation to reduce the circular detection complexity. For continuous frame image, the credibility based feature-checking rule is adopted to improve the overall efficiency of the algorithm.This scheme was applied in immersive virtual reality experiment system for middle school. Experimental results show that the system is stable and has a good interactive experience.

Key words: data glove, gesture recognition, visual information fusion, human-computer interaction, feature-checking