首页 > 论文 > 光电工程 > 37卷 > 12期(pp:88-95)


Interactive Volumetric Three-dimensional Display Based on Dynamic Hand Gesture Control

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文


研究了动态手势的跟踪和识别算法,实现了基于动态手势控制的交互式体三维显示。体三维显示可在真实的三维空间中呈现具有物理深度的三维图像,并且所显示图像可从360° 范围内任意角度裸眼观看。采用基于视觉的动态手势控制,可从围绕显示系统的任意视角实现交互式体三维显示。首先从相邻帧帧差图像中检测特定的静态手势获得初始跟踪区域,并针对此区域建立手部肤色的分布模型。然后通过光流法更新跟踪区域内KLT 特征点的位置,并按特征点群算法结合手部肤色的分布模型对特征点的位置进行修正。对于跟踪获得的动态手势轨迹,经过坐标变换并按八方向Freeman 链码进行量化编码,获得手势观测序列。最后利用经过训练建立的手势隐马尔科夫模型库,实现动态手势的识别。将动态手势控制应用到交互式体三维显示并进行了实验。实验结果表明,手势跟踪和识别算法准确而快速,对不同手势的平均识别正确率超过93%,平均响应时间小于40 ms,能从任意视角较好地实现交互式体三维显示。


Tracking and recognition algorithms for dynamic hand gestures have been studied and interactive volumetric three-dimensional (3D) display based on dynamic hand gesture control has been implemented. Volumetric 3D display can present images with physical depth cues in the true 3D space, where images can be viewed from almost any angle without any special eyewear. With vision-based hand gesture control, viewers can interact with volumetric 3D display from any viewpoint. Firstly, tracking region containing a special static hand posture is initialized from frame-differencing images. Meanwhile, a skin color model is built based on the region. Then positions of KLT features in the initial region are updated using optical flow tracking method and adjusted with the “flock of features” algorithm and the learned skin color model. Thereafter, through coordinate transformation and encoding according to eight-directional Freeman chain code, gesture observation sequence is generated for recognition. Finally, a pre-specified gesture symbol is decided by Hidden Markov Model (HMM) evaluation using a trained gesture HMM set. Experiments are conducted for the interactive volumetric display based on dynamic hand gesture control. Experimental results show that with these gesture tracking and recognition algorithms, the average recognition rate is high up to 93%, the average response time is less than 40 ms, and interactions can be well performed from arbitrary viewpoint.




基金项目:国家863 资助项目(2007AA01Z338)




作者单位    点击查看

潘文平:南京航空航天大学 自动化学院,南京 210016
沈春林:南京航空航天大学 自动化学院,南京 210016
张赵行:中国科学院自动化研究所,北京 100190
邢建芳:南京航空航天大学 自动化学院,南京 210016



【1】Bundell B G,Schwarz J A. Volumetric Three-Dimensional Display Systems [M]. John Wiley&Sons,2000.

【2】Gong X,Kirk M,Chu J,et al. Evaluation of a Volumetric Display for Radiation Therapy Treatment Planning [J]. Medical Physics(S0094-2405),2006,33(6):2209-2211.

【3】Sturman D J,Zeltzer D. A Survey of Glove-based Input [J]. IEEE Computer Graphics and Applications(S0272-1716),1994,14(1):30-39.

【4】Wang R Y,Popovi J. Real-time hand-tracking with a color glove [J]. ACM Transactions on Graphics(S0730-0301),2009,28(3):63-1-63-8.

【5】Mistry P,Maes P,Chang L. WUW - wear Ur world: a wearable gestural interface [C]// Proceedings of the 27th international Conference Extended Abstracts on Human Factors in Computing Systems. CHI 2009,Boston,MA,USA,April 4-9,2009:4111-4116.

【6】Just A,Marcel S. A comparative study of two state-of-the-art sequence processing techniques for hand gesture recognition [J].Computer Vision and Image Understanding(S1077-3142),2009,113(4):532-543.

【7】LaViola Joseph J. Bringing VR and Spatial 3D Interaction to the Masses through Video Games [J]. IEEE Computer Graphics and Applications(S0272-1716),2008,28(5):10-15.

【8】Charbonneau E,Miller A,Wingrave C,et al. Understanding visual interfaces for the next generation of dance-based rhythm video games [C]// Proceedings of the 2009 ACM SIGGRAPH Symposium on Video Games. Sandbox ''09,New Orleans,Louisiana,USA,August 3-7,2009:119-126.

【9】陈一民,张云华. 基于手势识别的机器人人机交互技术研究[J]. 机器人,2009,31(4):351-356.
CHEN Yi-min,ZHANG Yun-hua. Research on Human-Robot Interaction Technique Based on Hand Gesture Recognition [J].Robot(S1002-0446),2009,31(4):351-356.

【10】Kolsch M,Turk M. Robust Hand Detection [C]// Proceeding of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition. FGR 2004,Seoul,Korea,May 17-19,2004:614-619.

【11】Kolsch M,Turk M. Fast 2D Hand Tracking with Flocks of Features and Multi-Cue Integration [C]// Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop. CVPRW ''04,Washington,USA,June 27-July 02,2004:158.

【12】Bretzner L,Laptev I,Lindeberg T. Hand Gesture Recognition using Multi-Scale Color Features, Hierarchical Models and Particle Filtering [C]// Proceedings of the Fifth IEEE international Conference on Automatic Face and Gesture Recognition(FGR 2002),Washington,DC,USA,May 20-21,2002:423-428.

【13】SHI J B,Tomasi C. Good features to track [C]// Proceeding of IEEE Conference on Computer Vision and Pattern Recognition(CVPR 1994),Seattle,USA,June 21-23,1994:593-600.

【14】Martinez Aleix. Face image retrieval using HMMs [C]// Proceedings of the IEEE Workshop on Content-Based Access of Image and Video Libraries(CBAIVL 1999),Fort Collins,Colorado,USA,July 20-22,1999:35-39.

【15】Knut Langhans. New portable FELIX 3D display [J]. Proc. of SPIE(S0277-786X),1998,3296:204-216.

【16】Favalora G E,Dorval R K,Hall D M,et al. Volumetric three dimensional display system with rasterization hardware [J]. Proc.of SPIE(S0277-786X),2001,4297:227-235.

【17】Sullivan Alan. A Solid-state Multi-planar Volumetric Display [J]. SID Symposium Digest of Technical Papers(S0743-1562),2003,34(1):1531-1533.

【18】林远芳,刘向东,刘旭,等. 基于二维旋转屏的体三维显示系统像素属性分析[J]. 光子学报,2004,33(4):476-487.LIN Yuan-fang,LIU Xiang-dong,LIU Xu,et al. The analysis of voxel attributes in three-dimensianal volumetric display system based on the rotation of a two-dimensional display panel [J]. Acta Photonica Sinica,2004,33(4):476-487.

【19】李莉,李玉峰,沈春林,等. 基于数字微镜的旋转体三维显示装置研究 [J]. 仪器仪表学报,2008,29(1):67-72.LI Li,LI Yu-feng,SHEN Chun-lin,et al. Volumetric 3D display device based on DMD utilizing a rotating screen [J]. Chinese Journal of Scientific Instrument (S0254-3087),2008,29(1):67-72.

【20】潘文平,沈春林,李莉,等. 基于数字微镜器件和旋转螺旋屏的体三维显示系统[J]. 光电工程,2010,37(10):144-150.PAN Wen-ping,SHEN Chun-lin,LI Li,et al. Volumetric Three-dimensional Display System Based on DMD and A Spinning Helical Screen [J]. Opto-Electronic Engineering,2010,37(10):144-150.

【21】Kim Daehwan,Song Jinyoung,Kim Daijin. Simultaneous gesture segmentation and recognition based on forward spotting accumulative HMMs [J]. Pattern Recognition(S0031-3203),2007,40:3012-3026.

【22】Grossman T,Wigdor D,Balakrishnan R. Multi-Finger Gestural Interaction with 3D Volumetric Displays [C]// Proceedings of the 17th Annual ACM Symposium on User interface Software and Technology(UIST ''04),Santa Fe,NM,USA,October 24-27,2004:61-70.


PAN Wen-ping,SHEN Chun-lin,ZHANG Zhao-xing,XING Jian-fang. Interactive Volumetric Three-dimensional Display Based on Dynamic Hand Gesture Control[J]. Opto-Electronic Engineering, 2010, 37(12): 88-95

潘文平,沈春林,张赵行,邢建芳. 基于动态手势控制的交互式体三维显示[J]. 光电工程, 2010, 37(12): 88-95


【1】彭祎帆,陶毅阳,于超,李海峰,刘旭. 基于红外结构光的三维显示用交互装置. 光学学报, 2013, 33(4): 412005--1

【2】王国庆,桂进斌,姜智翔,金晓宇. 交互式全息显示进展. 激光与光电子学进展, 2019, 56(8): 80004--1

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF