Three-dimensional (3D) range-intensity correlation laser imaging based on two gate images is a novel 3D reconstruction technique^{[1–5" target="_self" style="display: inline;">–5]} and has great potential in 3D real-time imaging applications of underwater environment survey^[6], 3D surveillance^[7,8], marine fish, and plankton in situ detection^[9]. In this technique, a 3D scene can be reconstructed by range-intensity correlation of two overlapped gate images captured by a range-intensity correlation laser imaging system (RICLIS), as shown in Fig. 1(a). A typical RICLIS mainly consists of a pulsed laser illuminator, an intensified CCD (ICCD) camera, and a timing control unit (TCU). Up to now, this technique has been mainly used for 3D imaging of static scenes or quasi-static targets, where the common areas of two consecutive frame images match properly^[10,11]. However, for moving targets or platforms, such as pan–tilt device, airborne, ship, underwater remote operated vehicle, and autonomous underwater vehicle, image misalignment due to motion causes the mismatching of common areas, as depicted in Fig. 1(a), and causes a large error or failure of 3D reconstruction by the triangular or trapezoidal algorithm^[12]. As shown in Fig. 1(b), the target of motion leads to a failure in 3D reconstruction. In order to overcome this drawback, we propose a method of coordinate difference homogenization (CDH) matching for compensating the image’s misalignment due to motion and making the common area match properly.

Fig. 1. (Color online) (a) Method of 3D RICLIS. (b) 3D images of motion and static scenes.

下载图片 查看所有图片

The process of the CDH matching method is shown in Fig. 2. In the method, stable and robust features are obtained by the speeded-up robust features (SURFs)^[13], and the features are matched by bi-directional feature matching based on the Euclidean distance between the feature vectors. The bi-directional feature matching method selects common feature pairs from the two unidirectional matching (A–B and B–A) to get robust feature pairs, as depicted in Figs. 3(a)–3(c). Then, the CDH is used to optimize the feature pairs, and homography is calculated by the random sample consensus (RANSAC) algorithm^[14] to process the geometrical transformation that relates the images. Finally, an image geometrical rectification is proposed to transform the two images.

Fig. 2. Process of the CDH matching method.

下载图片 查看所有图片

Fig. 3. (Color online) (a) A–B unidirectional matching. (b) B–A unidirectional matching. (c) Robust feature pairs. (d) The principle of the CDH method.

下载图片 查看所有图片

The principle of the CDH is depicted in Fig. 3(d). Gate images A and B are outputted by an RICLIS. The positions of the target tree are different in the two gate images due to the motion influence. Feature pairs F1 and F1′ have different coordinates (u1,v1) and (u1′,v1′), feature pairs F2 and F2′ have different coordinates (u2,v2) and (u2′,v2′), and coordinate differences (u1−u1′,v1−v1′) between F1 and F1′ are different from (u2−u2′,v2−v2′) between F2 and F2′. The process of the CDH method is as follows: step one, calculate the original mean value (Δu−,Δv−) of the coordinate differences, as shown in (Δu−,Δv−)=(∑i=1I(ui−ui′)I,∑i=1I(vi−vi′)I),(1)where I is the number of the feature pairs, i is the ith feature pair. Step two, remove the wrong feature pairs by comparing the original mean value (Δu−,Δv−) with the coordinate differences, if |(ui−ui′)−Δu−|≤t and |(vi−vi′)−Δv−|≤t, keep the feature pairs, or else I and (Δu−,Δv−) are updated, t is an experience threshold. Step three, the final feature coordinates are re-registered from (ui′,vi′) to (ui′′,vi′′) based on the updated optimized mean value with (ui′′,vi′′)=(ui−Δu−,vi−Δv−).(2)

For experimental research, an RICLIS is established by a pulsed laser, a gated ICCD, and a TCU. The system is on a pan–tilt device to scan targets. The laser is a laser diode with a center wavelength of 808 nm, and its laser pulse width can be changed from 100 ns to several microseconds under the trigger of the TCU. For the gated ICCD, a gated GEN II intensifier is coupled to a CCD with 1392×1040pixels, and the pixel size in the image sensor chip size is 6.45μm×6.45μm. The maximal repetition frequency is 100 kHz, and the minimal gate time is 40 ns. The TCU realized by the field-programmable gate array (FPGA) can provide the desired time sequence for the pulsed laser and the gated ICCD. In the experiment, the target is a communication tower at the location of about 780 m in Fig. 4, we obtain two gate images of the tower, the laser pulse width is 500 ns, the peak power is 100 W, and the time delays are 5100 and 5600 ns, respectively. The gate width is 500 ns, and the CCD has a frame rate of 15 frames per second with an exposure time of 40 ms. The two images are mismatched due to the motion of the pan–tilt device. The pan–tilt rotates 3 deg/s.

Fig. 4. RICLIS and the target of tower.

下载图片 查看所有图片

Figures 5(a), 5(d), and 5(g) are gate images A with the time delay of 5100 ns, Figs. 5(b), 5(e), and 5(h) are gate images B with the time delay of 5600 ns, and all of the images are enhanced for human eyes with the image enhancement method in Ref. [15]. Images of Figs. 5(c), 5(f), and 5(i) are 3D images of regions in the yellow line of the corresponding gate images. Figures 5(a) and 5(b) are the motion gate images under the motion condition of the pan–tilt device, Figs. 5(d) and 5(e) are the rectified gate images with the proposed method, and Figs. 5(g) and 5(h) are static gate images without imaging system motion.

Fig. 5. (Color online) Experimental results of a tower. (a)–(c) Motion gate images and 3D image. (d)–(f) Rectified gate images and 3D image. (g)–(i) Static gate images and 3D image.

下载图片 查看所有图片

As shown in Fig. 5(c), the 3D reconstruction of the tower fails due to the mismatch of the tower of two gate images. By the proposed method, the mismatched images are rectified, the 3D tower is successfully obtained, and the mean range value of the tower is 781.2 m, as shown in Fig. 5(f). The 3D result of the static gate images without motion is in Fig. 5(i), and the mean range value of the tower is 781.3 m.

Figure 6(a) shows the ranges of the red lines corresponding to the 800th row in Figs. 5(c), 5(f), and 5(i), and the mean range values of curves in the sub-graph are 37, 773.2, and 773.7 m, respectively. The slight deviation of ranges between the rectified and static images is caused by the differences of gray values, as shown in Fig. 6(b), the curves represent gray values of the red line at the 800th row in Figs. 5(d) and 5(g), and the mean gray values in the sub-graph are 18.6 and 17.2, respectively. Except for the atmosphere influence, the deviation of gray values is mainly caused by the non-uniformity of the laser, since the positions of the tower in Figs. 5 (b), 5(e), and 5(h) are the same, while the positions of the tower in Figs. 5(a) and 5(g) are different, and the gray values of Figs. 5(a) and 5(d) are the same. The non-uniformity illumination can be homogenized by homogenization techniques, like in Ref. [16].

Fig. 6. (Color online) (a) Comparison of range accuracy of red lines in Figs. 5(c), 5(f), and 5(i). (b) Gray trace of red lines in Figs. 5(d) and 5(g).

下载图片 查看所有图片

In conclusion, we propose a CDH matching method to solve the influence of motion in range-intensity correlation laser imaging. In the CDH method, the coordinates of feature pairs are optimized by using the mean value of coordinate differences, and the images with motion influence are rectified. With the proposed method, a 3D image of the tower at 780 m is obtained with motion correction, and two motion gate images are captured by a pan–tilt device. The results show that the method is available for motion correction due to moving targets or platforms in 3D range-intensity correlation laser imaging. Suppressing the influence of non-uniformity of the laser needs a further study in our future work.