首页 > 论文 > 光学学报 > 39卷 > 2期(pp:215004--1)

基于共同视域的自监督立体匹配算法

Self-Supervised Stereo Matching Algorithm Based on Common View

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

提出了一种基于共同视域的自监督立体匹配算法, 该算法根据视差的左右一致性来确定双目图像的共同可视区域, 从而抑制被遮挡区域产生的噪声, 为网络模型的学习提供了更加准确的反馈信号。研究结果表明:在没有任何标签数据的前提下, 所提算法的预测误差降低了11%~42%, 且与有监督立体匹配算法的性能相当。

Abstract

A self-supervised stereo matching algorithm is proposed based on common view. In this algorithm, the common visible region of the binocular images is determined according to the left-right consistency of disparity and thus the noise generated in the occluded region is suppressed, which provides more accurate feedback signals for the network model learning. The research results show that the prediction error of the proposed algorithm can be reduced by 11%-42% without any label data, and the performance of the proposed algorithm is comparable to that of the supervised stereo matching algorithm.

Newport宣传-MKS新实验室计划
补充资料

中图分类号:TP183

DOI:10.3788/AOS201939.0215004

所属栏目:机器视觉

收稿日期:2018-07-20

修改稿日期:2018-09-27

网络出版日期:2018-10-10

作者单位    点击查看

王玉锋:海军航空大学航空作战勤务学院, 山东 烟台 264001空军航空大学航空作战勤务学院, 吉林 长春 130022
王宏伟:空军航空大学飞行研究所, 吉林 长春 130022
吴晨:海军航空大学航空作战勤务学院, 山东 烟台 264001
刘宇:空军航空大学航空作战勤务学院, 吉林 长春 130022
袁昱纬:中国人民解放军91977部队, 北京102200
全吉成:海军航空大学航空作战勤务学院, 山东 烟台 264001空军航空大学航空作战勤务学院, 吉林 长春 130022

联系人作者:全吉成(jicheng_quan@126.com); 王玉锋(wangyf_1991@163.com);

【1】Liu C, Yuen J, Torralba A. SIFT flow: dense correspondence across scenes and its applications[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5): 978-994.

【2】Liu Y F, Cai Z J. Binocular stereo vision three-dimensional reconstruction algorithm based on ICP and SFM[J]. Laser and Optoelectronics Progress, 2018, 55(9): 091503.
刘一凡, 蔡振江. 基于ICP与SFM的双目立体视觉三维重构算法[J]. 激光与光电子学进展, 2018, 55(9): 091503.

【3】Sivaraman S, Trivedi M M. A review of recent developments in vision-based vehicle detection[C]. IEEE Intelligent Vehicles Symposium(IV), 2013: 310-315.

【4】Schmid K, Tomic T, Ruess F, et al. Stereo vision based indoor/outdoor navigation for flying robots[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013: 3955-3962.

【5】Scharstein D, Szeliski R, Zabih R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J]. International Journal of Computer Vision, 2002, 47(1): 7-42.

【6】Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.

【7】Dalal N, Triggs B. Histograms of oriented gradients for human detection[C]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2005: 886-893.

【8】Hirschmuller H. Stereo processing by semiglobal matching and mutual information[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2): 328-341.

【9】Liu D W, Han L, Han X Y. High spatial resolution remote sensing image classification based on deep learning[J]. Acta Optica Sinica, 2016, 36(4): 0428001.
刘大伟, 韩玲, 韩晓勇. 基于深度学习的高分辨率遥感影像分类研究[J]. 光学学报, 2016, 36(4): 0428001.

【10】Hou Y Q Y, Quan J C, Wei Y M. Valid aircraft detection system for remote sensing images based on cognitive models[J]. Acta Optica Sinica, 2018, 38(1): 0111005.
侯宇青阳, 全吉成, 魏湧明. 基于认知模型的遥感图像有效飞机检测系统[J]. 光学学报, 2018, 38(1): 0111005.

【11】Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651.

【12】bontar J, LeCun Y. Computing the stereo matching cost with a convolutional neural network[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015: 1592-1599.

【13】bontar J, LeCun Y. Stereo matching by training a convolutional neural network to compare image patches[J]. Journal of Machine Learning Research, 2016, 17(1): 2287-2318.

【14】Xiao J S, Tian H, Zou W T, et al. Stereo matching based on convolutional neural network[J]. Acta Optica Sinica, 2018, 38(8): 0815017.
肖进胜, 田红, 邹文涛, 等. 基于深度卷积神经网络的双目立体视觉匹配算法[J]. 光学学报, 2018, 38(8): 0815017.

【15】Chen Z Y, Sun X, Wang L, et al. A deep visual correspondence embedding model for stereo matching costs[C]. IEEE International Conference on Computer Vision, 2015: 972-980.

【16】Park H, Lee K M. Look wider to match image patches with convolutional neural networks[J]. IEEE Signal Processing Letters, 2017, 24(12): 1788-1792.

【17】Güney F, Geiger A. Displets: resolving stereo ambiguities using object knowledge[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015: 4165-4175.

【18】Seki A, Pollefeys M. Patch based confidence prediction for dense disparity map[C]. British Machine Vision Conference (BMVC), 2016: 23.

【19】Mayer N, Ilg E, Husser P, et al. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2016: 4040-4048.

【20】Pang J H, Sun W X, Ren J S, et al. Cascade residual learning: a two-stage convolutional neural network for stereo matching[C]. IEEE International Conference on Computer Vision Workshops, 2017: 887-895.

【21】Liang Z F, Feng Y L, Guo Y L, et al. Learning deep correspondence through prior and posterior feature constancy[EB/OL]. (2017-12-04)[2018-06-05]. http://cn.arxiv.org/abs/1712.01039.

【22】Kendall A, Martirosyan H, Dasgupta S, et al. End-to-end learning of geometry and context for deep stereo regression[C]. IEEE International Conference on Computer Vision, 2017: 66-75.

【23】Yu L D, Wang Y C, Wu Y W, et al. Deep stereo matching with explicit cost aggregation sub-architecture[EB/OL]. (2018-01-12)[2018-06-05]. http://cn.arxiv.org/abs/1801.04065.

【24】Chang J R, Chen Y S. Pyramid stereo matching network[EB/OL]. (2018-03-23)[2018-06-05]. http://cn.arxiv.org/abs/1803.08669.

【25】Xie J Y, Girshick R, Farhadi A. Deep 3D: fully automatic 2D-to-3D video conversion with deep convolutional neural networks[C]. European Conference on Computer Vision(ECCV), 2016: 842-857.

【26】GargR, Vijay K B G, Carneiro G, et al. Unsupervised CNN for single view depth estimation: Geometry to the rescue[C]. European Conference on Computer Vision (ECCV), 2016: 740-756.

【27】Godard C, Aodha O M, Brostow G J. Unsupervised monocular depth estimation with left-right consistency[C]. IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2017: 6602-6611.

【28】Jaderberg M, Simonyan K, Zisserman A, et al. Spatial transformer networks[EB/OL]. (2015-06-05)[2018-06-05]. http://cn.arxiv.org/abs/1506.02025.

【29】Ye M L, Johns E, Handa A, et al. Self-supervised siamese learning on stereo image pairs for depth estimation in robotic surgery[EB/OL]. (2017-05-17)[2018-06-05]. http://cn.arxiv.org/abs/1705.08260.

【30】LiuX, Sinha A, Unberath M, et al. Self-supervised learning for dense depth estimation in monocular endoscopy[C]. European Conference on Computer Vision (ECCV), 2018, 11041: 128-138.

【31】Zhong Y R, Dai Y C, Li H D. Self-supervised learning for stereo matching with self-improving ability[EB/OL]. (2017-09-04)[2018-06-05]. http://cn.arxiv.org/abs/1709.00930.

【32】Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.

【33】Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving The KITTI vision benchmark suite[C]. IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2012: 3354-3361.

【34】Menze M, Geiger A. Object scene flow for autonomous vehicles[C]. IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2015: 3061-3070.

【35】wyf2017: Pytorch implementation of the several deep stereo matching network [EB/OL]. https://github.com/wyf2017/DSMnet.

【36】Kingma D P, Ba J. Adam: a method for stochastic optimization[EB/OL]. (2014-12-22)[2018-06-05]. http://cn.arxiv.org/abs/1412.6980.

引用该论文

Wang Yufeng,Wang Hongwei,Wu Chen,Liu Yu,Yuan Yuwei,Quan Jicheng. Self-Supervised Stereo Matching Algorithm Based on Common View[J]. Acta Optica Sinica, 2019, 39(2): 0215004

王玉锋,王宏伟,吴晨,刘宇,袁昱纬,全吉成. 基于共同视域的自监督立体匹配算法[J]. 光学学报, 2019, 39(2): 0215004

被引情况

【1】王玉锋,王宏伟,于光,杨明权,袁昱纬,全吉成. 基于三维卷积神经网络的立体匹配算法. 光学学报, 2019, 39(11): 1115001--1

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF