光电工程, 2019, 46 (12): 190006, 网络出版: 2020-01-09   

基于深度学习的真实尺度运动恢复结构方法

Equal-scale structure from motion method based on deep learning
作者单位
浙江工业大学信息工程学院,浙江 杭州 310023
摘要
传统的多视图几何方法获取场景结构存在两个问题:一是因图片模糊和低纹理带来的特征点误匹配,从而导致重建精度降低;二是单目相机缺少尺度信息,重建结果只能确定未知的比例因子,无法获取准确的场景结构。针对这些问题本文提出一种基于深度学习的真实尺度运动恢复结构方法。首先使用卷积神经网络获取图片的深度信息;接着为了恢复单目相机的尺度信息,引入惯性传感单元(IMU),将IMU获取的加速度和角速度与ORB-SLAM2获取的相机位姿进行时域和频域上的协同,在频域中获取单目相机的尺度信息;最后将图片的深度图和具有尺度因子的相机位姿进行融合,重建出场景的三维结构。实验表明,使用Depth CNN网络获取的单目图像深度图解决了多层卷积池化操作输出图像分辨率低和缺少重要特征信息的问题,绝对值误差达到了0.192,准确率高达0.959;采用多传感器融合的方法,在频域上获取单目相机的尺度能够达到0.24 m的尺度误差,相比于VIORB方法获取的相机尺度精度更高;重建的三维模型与真实大小具有0.2 m左右的误差,验证了本文方法的有效性。
Abstract
Two problems exist in traditional multi-view geometry method to obtain the three-dimensional structure of the scene. First, the mismatching of the feature points caused by the blurred image and low texture, which reduces the accuracy of reconstruction; second, as the information obtained by monocular camera is lack of scale, the reconstruction results can only determine the unknown scale factor, and cannot get accurate scene structure. This paper proposes a method of equal-scale motion restoration structure based on deep learning. First, the convolutional neural network is used to obtain the depth information of the image; then, to restore the scale information of the monocular camera, an inertial measurement unit (IMU) is introduced, and the acceleration and angular velocity acquired by the IMU and the camera position acquired by the ORB-SLAM2 are demonstrated. The pose is coordinated in both time domain and frequency domain, and the scale information from the monocular camera is acquired in the frequency domain; finally, the depth information of the image and the camera pose with the scale factor are merged to reconstruct the three-dimensional structure of the scene. Experiments show that the monocular image depth map obtained by the Depth CNN network solves the problem that the output image of the multi-level convolution pooling operation has low resolution and lacks important feature information, and the absolute value error reaches 0.192, and the accuracy rate is up to 0.959. The multi-sensor fusion method can achieve a scale error of 0.24 m in the frequency domain, which is more accurate than that of the VIORB method in the frequency domain. The error between the reconstructed 3D model and the real size is about 0.2 m, which verifies the effectiveness of the proposed method.

陈朋, 任金金, 王海霞, 汤粤生, 梁荣华. 基于深度学习的真实尺度运动恢复结构方法[J]. 光电工程, 2019, 46(12): 190006. Chen Peng, Ren Jinjin, Wang Haixia, Tang Yuesheng, Liang Ronghua. Equal-scale structure from motion method based on deep learning[J]. Opto-Electronic Engineering, 2019, 46(12): 190006.

本文已被 2 篇论文引用
被引统计数据来源于中国光学期刊网
引用该论文: TXT   |   EndNote

相关论文

加载中...

关于本站 Cookie 的使用提示

中国光学期刊网使用基于 cookie 的技术来更好地为您提供各项服务,点击此处了解我们的隐私策略。 如您需继续使用本网站,请您授权我们使用本地 cookie 来保存部分信息。
全站搜索
您最值得信赖的光电行业旗舰网络服务平台!