1. Introduction
Pressure piping serves as a vital channel for transporting materials such as oil, natural gas, and chemicals. The three-dimensional reconstruction of pressure piping can provide geometric information about pipelines as an important basis for pipeline design, maintenance, and safety assessment.
Recently, with the rapid development of three-dimensional laser radar technology, this technology has achieved great success in unmanned driving [Reference Liu, Di and Xu1, Reference Zhang, Wang, Jiang, Zeng and Dai2], remote sensing mapping [Reference Ma, Zhou, Wen, Gen, Huang, Wu and Pei3], robot navigation and positioning [Reference Sarkar, Prabhakar and Ghose4], and other fields [Reference Diab, Kashef and Shaker5–Reference Cheng, Sun and Meng10]; examples of specific applications include the creation of high-precision maps in unmanned driving, the rapid acquisition of digital elevation models in remote sensing mapping, and navigation and obstacle avoidance in mobile robots. Moreover, three-dimensional reconstruction based on laser point clouds is a popular topic of current research [Reference Chen, Wu and He11–Reference Ou, Hong, Kyzer, Yang, Zhou and Wang15]. Scholars in China and elsewhere have conducted much-related research on this subject. Some representative programs include Bundle Adjustment for Lidar Mapping (BALM) [Reference Liu and Zhang16], tightly coupled laser-inertial odometry and mapping with bundle adjustment (BA-LIOM) [Reference Li, Zhang, Zhang, Yuan, Liu and Wu17], Lightweight and Ground-Optimized Lidar Odometry and Mapping on Variable Terrain (LeGO-LOAM) [Reference Shan and Englot18], Direct LiDAR Odometry (DLO) [Reference Fasiolo, Scalera and Maset19], Lidar Inertial Odometry Via Smoothing And Mapping (LIO-SAM) [Reference Shan, Englot, Meyers, Wang, Ratti and Rus20], Fast Direct LiDAR-Inertial Odometry (FAST-LIO2) [Reference Xu, Cai, He, Lin and Zhang21], and Lightweight Tightly Coupled Lidar-Inertial Odometry Using Parallel Sparse Incremental Voxels (Faster-LIO) [Reference Bai, Xiao, Chen, Wang, Zhang and Gao22]. However, although these programs can achieve very good results in laboratory environments and on public datasets, they often fail easily for scenes with sparse textures and features [Reference Huang, Mei, Zhang and Abbas12]. This is because most of the existing three-dimensional reconstruction methods rely on the approach of extracting points, lines, surfaces, and other features to achieve point cloud registration, which is not suitable for such scenes; consequently, these methods cannot meet practical needs [Reference Sarkar, Prabhakar and Ghose4].
The point cloud registration algorithm is the core technology of three-dimensional reconstruction and thus has an important impact on the accuracy of the reconstruction results. Therefore, research on point cloud registration technology for scenes with sparse textures and features is the key to the three-dimensional reconstruction of such scenes [Reference Yang, Cheng and Wang7, Reference Liu, Sun and Dong13]. However, there is currently relatively little research on the point cloud registration issues that arise in scenarios with sparse features and textures. This is mainly because, in such scenarios, the feature and texture information contained in the point cloud data is limited or nonexistent, posing significant challenges for registration algorithms [Reference Cheng, Sun and Meng10].
To solve the problem of the three-dimensional reconstruction of pressure piping in special industrial scenarios, this paper proposes a fitting-based point cloud registration algorithm with cylinder axis vector constraints. By using certain geometric parameters of the pressure piping to be modeled as constraints and incorporating them into the traditional iterative closest point (ICP) algorithm, three-dimensional reconstruction of pressure piping with sparse textures and features can be achieved.
The main contributions of this article include the following:
-
To address the three-dimensional reconstruction problem for cylindrical or cylinder-like pipelines, an improved ICP algorithm is proposed that relies on point cloud fitting with constraint conditions on the cylinder axis vectors.
-
For scenes with environmental interference, incorporating the proposed point cloud registration algorithm into the front-end part of the 3D reconstruction algorithm can improve the accuracy of 3D reconstruction under certain conditions.
-
A multifunctional dual-track wall-climbing robot system has been designed to achieve efficient detection of pressure pipelines to support their intelligent operation and maintenance.
This paper is organized as follows. In Section 2, related work regarding point cloud registration is presented. The principle of the improved fitting-based ICP algorithm with cylinder axis vector constraints is described in Section 3. Experimental studies are introduced in Section 4, and the experimental results are discussed in Section 5. Finally, in Section 6, the conclusion and plans for future work regarding the proposed method are presented.
2. Related work
Accurate point cloud registration is the key to three-dimensional reconstruction. At present, the most widely used algorithm for laser point cloud registration is the ICP algorithm [Reference Chen and Medioni23]. There are also many optimized variants of the ICP algorithm, such as PL-ICP [Reference Censi24], NICP [Reference Serafin and Grisetti25], GICP [Reference Koide, Yokozuka, Oishi and Banno26], Robust ICP [Reference Zhang, Yao and Deng27], Sparse ICP [Reference Bouaziz, Tagliasacchi and Pauly28], AA-ICP [Reference Pavlov, Ovchinnikov, Derbyshev, Tsetserukou and Oseledets29], Symmetric ICP [Reference Rusinkiewicz30], and Go-ICP [Reference Yang, Li, Campbell and Jia31]. The basic idea of the classical ICP algorithm is to construct an error function in accordance with the correspondence relationships between point pairs in two frame point clouds and then obtain the optimal pose transformation parameters by minimizing the error function. Other improved algorithms based on ICP include the addition of constraints when constructing the error function and the adoption of different optimization algorithms when minimizing the error function. The disadvantage of the ICP algorithm is that it depends heavily on the initial values and can easily fall into a locally optimal solution. When the initial values are not suitable, the number of iterations will increase, and with sufficiently large initial errors, incorrect results may be obtained. Moreover, the ICP algorithm exhibits first-order convergence, and the convergence speed is slow. The ICP algorithm is ideal for the registration of point sets with distinctive local characteristics. However, for spheres, cylinders, cubes, and other regular geometric shapes, the classical ICP algorithm cannot achieve convergence because it cannot correctly determine the corresponding point of any particular point on such a shape.
Other point cloud registration algorithms based on the Gaussian hypothesis include normal distribution transformation (NDT) [Reference Magnusson32] and coherent point drift (CPD) [Reference Myronenko and Song33]. The basic idea of the NDT algorithm is to use the characteristics of a Gaussian distribution to construct a multidimensional normal distribution based on the reference point cloud and then determine the pose parameters that maximize the probability density of the converted point cloud in the reference point cloud coordinate system. The pose parameters obtained in this way are considered the optimal pose parameters. Variants of the NDT algorithm include The three-dimensional normal-distributions transform (3D-NDT) [Reference Stoyanov, Magnusson, Andreasson and Lilienthal34], Point-to-Distribution Normal Distributions Transform (P2D-NDT) [Reference Magnusson, Vaskevicius, Stoyanov, Pathak and Birk35], and Distribution-to-Distribution Normal Distributions Transform (D2D-NDT) [Reference Magnusson, Vaskevicius, Stoyanov, Pathak and Birk35]. The advantage of the NDT algorithm is that it can process large point cloud datasets; however, it needs to convert the original point cloud data into a Gaussian distribution, and its calculation speed is relatively slow.
The CPD algorithm converts the point set registration problem into a probability density estimation problem. The advantages of the CPD algorithm are that it can handle both rigid and nonrigid transformations and that it performs well for the registration of partially overlapping or noisy point clouds. However, this algorithm is nevertheless sensitive to noise, and its registration effect is not ideal for scenes with uncertain noise. Super4PCS [Reference Mellado, Aiger and Mitra36] is a point cloud registration method that does not require an initial pose, exhibits good robustness, and has a wide range of applications. However, this method is slow, has low accuracy, and requires multiple iterations. TEASER [Reference Yang, Shi and Carlone37] is an algorithm for three-dimensional point cloud registration that is suitable for situations with many outliers. It uses a truncated least squares cost function to reformulate the registration problem, making the pose estimation less sensitive to many false corresponding points. This algorithm also runs very fast. However, the introduction of the truncated least squares function can lead to nonconvex optimization problems that are difficult to solve and can easily become trapped in local optima.
Another type of model-free point cloud registration algorithm is based mainly on machine learning or deep learning. The basic idea of deep learning-based point cloud registration algorithms is to use an end-to-end neural network to transform the registration problem into a regression problem. The input provided to the neural network is the two frame point clouds to be registered, and the output is the transformation matrix between the two frame point clouds. Representative programs include GeoTransformer [Reference Qin, Yu, Wang, Guo, Peng and Xu38], CoFiNet [Reference Yu, Li, Saleh, Busam and Ilic39], Predator [Reference Huang, Gojcic, Usvyatsov, Wieser and Schindler40], Efficient LO-Net [Reference Wang, Wu, Jiang, Liu and Wang41], REGTR [Reference Yew and Lee42], and RGM [Reference Fu, Liu, Luo and Wang43]. The performance and speed of these deep learning models on test datasets are far better than those of the ICP and NDT algorithms. However, because the estimation of the transformation parameters is regarded as a black box in the regression process, the distances are measured in a coordinate-based Euclidean space and thus are sensitive to noise. Moreover, methods based on deep learning require data to be collected in advance and labeled to some extent, so they are not suitable for the three-dimensional reconstruction of unknown environments.
To address the problems encountered by the point cloud registration algorithms mentioned above in scenarios with sparse features and textures, this paper proposes a novel point cloud registration method. The proposed method can solve the problem of point cloud registration when textures and features are scarce. Through practical testing, the effectiveness of the proposed point cloud registration algorithm is verified.
3. Point cloud registration algorithm based On fitting with cylinder axis vector constraints
The complete execution process of the improved point cloud registration algorithm based on point cloud fitting with cylinder axis vector constraints is shown in Fig. 1. The technical details are described in detail below. For two overlapping frame point clouds
$P = \left \{{{p_i}|{p_i} \in{R^3},i = 1,2, \ldots m} \right \}$
and
$Q = \left \{{{q_j}|{q_j} \in{R^3},j = 1,2, \ldots n} \right \}$
in Euclidean space, the basic principle of the classical ICP algorithm is to achieve the registration of
$P$
and
$Q$
through rigid transformation. Let the transformation matrices be denoted by
$\left [{R,t} \right ]$
; then, the function for transforming the point cloud
$P$
into the point cloud
$Q$
via the transformation matrices
$\left [{R,t} \right ]$
is denoted by
$F\left ({R,t} \right )$
, and the optimal transformation matrices can be obtained by solving
$\min \left ({F\left ({R,t} \right )} \right )$
. Here,
$F\left ({R,t} \right )$
can be expressed as shown in Eq. 1. PL-ICP is based on a constraint relationship constructed in terms of the distance from a point to a line and obtains the optimal pose transformation matrices by minimizing the point-to-line distance. The error function
$E\left ({R,t} \right )$
of the point-to-line distance in the PL-ICP algorithm is expressed as shown in Eq. 2, where
$n_i$
represents the normal vector of the line on which
$q_i$
is located. Although the PL-ICP algorithm converges faster than the ICP algorithm, it also falls into local extrema more easily.
\begin{equation} F\left ({R,t} \right ) ={\sum \limits _{i = 1}^n{||R{p_i} + t -{q_i}||} ^2} \end{equation}
\begin{equation} E\left ({R,t} \right ) = \frac{1}{n}\sum \limits _{i = 1}^{n }{{{\left ({\left ({{p_i} - \left ({R{q_i} + t} \right )} \right ) \cdot{n_i}} \right )}^2}} \end{equation}
Inspired by the above registration algorithm in combination with the geometric characteristics of the point cloud to be registered, we consider an approach in which the point cloud to be registered is first fit to a certain geometric shape, and then point cloud registration is realized based on similarity to that geometric shape. This method can reduce the computational burden of point cloud registration to some extent while improving the accuracy of registration. Here, the algorithm proposed in this paper is described by taking the three-dimensional reconstruction of a penstock as an example. In accordance with the distinctive geometric characteristics of a penstock, it can be considered to be composed of several cylinder-like components. Therefore, the penstock point cloud data collected via three-dimensional lidar can be theoretically fit to a cylindrical geometry. The specific fitting method used can be the least squares method.
Figure 1. Complete technical roadmap of the improved point cloud registration algorithm based on point cloud fitting with cylinder axis vector constraints.
According to the geometric characteristics of a cylinder, the distance from any point
$A\left ({x,y,z} \right )$
on the surface of a cylinder to its axis is equal to its radius
$r$
. A cylinder can be uniquely determined by any point
${M_0}\left ({{x_0},{y_0},{z_0}} \right )$
on the axis of the cylinder and the cylinder axis vector
$V\left ({l,s,k} \right )$
. The equation describing a cylinder is given in Eq. 3.
\begin{equation} \begin{array}{l}{\left ({x -{x_0}} \right )^2} +{\left ({y -{y_0}} \right )^2}{\rm{ + }}{\left ({z -{z_0}} \right )^2} - \\{[l(x -{x_0}) + s(y -{y_0}) + k(z -{z_0})]^2} ={r^2} \end{array} \end{equation}
The first step of cylinder fitting is to determine the initial values of the cylinder model parameters, and the second step is to establish an error equation to solve for the optimal parameter values.
The initial estimate of the fitted cylinder axis vector,
${V^0}\left ({{l^0},{s^0},{k^0}} \right )$
, is obtained as follows. First, an arbitrary point on the surface of the cylinder is selected, and its adjacent points are used to perform local plane fitting to obtain the unit normal vector at that point. This process is repeated for a number of surface points, and then, principal component analysis is used to fit all of the unit normal vectors obtained in this way to a plane. Finally, the unit normal vector of the fitted plane is taken as the initial estimate of the cylinder axis vector.
Here, we assume that the initial cylinder axis vector is a unit vector pointing in the positive direction that satisfies Eq. 4, such that
${l_0} \geq 0$
; if
${l_0} = 0$
, then
${s_0} \gt 0$
; if
${l_0} = 0$
and
${s_0} = 0$
, then
${k_0} \gt 0$
. Notably,
$l_0$
,
$s_0$
, and
$k_0$
cannot be simultaneously zero here.
\begin{equation} l_0^2 + s_0^2 + k_0^2 = 1 \end{equation}
Based on the aforementioned axis vector constraints, Eq. 5 can be used to obtain the initial coordinates of any point on the cylinder axis.
\begin{equation} \left \{ \begin{gathered}{x_0} = \frac{{\sum \limits _{i = 1}^N{{x_i}} }}{N},\mathop{}\limits ^{} |l_0| \gt |s_0| \cap |l_0| \gt |k_0| \\{y_0} = \frac{{\sum \limits _{i = 1}^N{{y_i}} }}{N},\mathop{}\limits ^{} |s_0| \gt |l_0| \cap |s_0| \gt |k_0| \\{z_0} = \frac{{\sum \limits _{i = 1}^N{{z_i}} }}{N},\mathop{}\limits ^{} |k_0| \gt |l_0| \cap |k_0| \gt |s_0| \\ \end{gathered} \right . \end{equation}
Subsequently, an error function can be built based on the distance from any point on the surface of the cylinder to a point on its axis, as shown in Eq.6.
\begin{equation} \begin{array}{l} e ={\left ({x -{x_0}} \right )^2} +{\left ({y -{y_0}} \right )^2} +{\left ({z -{z_0}} \right )^2} - \\{\left [{{l}\left ({x -{x_0}} \right ) +{s}\left ({y -{y_0}} \right ) +{k}\left ({z -{z_0}} \right )} \right ]^2} -{r^2} \end{array} \end{equation}
By linearizing the equation for the error
$e$
, Eq. 7 is obtained.
\begin{equation} \begin{array}{l} e ={e_0} + \frac{{\partial e}}{{\partial{x}}}{x} + \frac{{\partial e}}{{\partial{y}}}{y} + \frac{{\partial e}}{{\partial{z}}}{z} + \frac{{\partial e}}{{\partial l}}l + \frac{{\partial e}}{{\partial s}}s + \frac{{\partial e}}{{\partial k}}k + \frac{{\partial e}}{{\partial r}}r \end{array} \end{equation}
where
$e_0$
can be obtained from Eq. 8 and
$\frac{{\partial e}}{{\partial{x}}}$
,
$\frac{{\partial e}}{{\partial{y}}}$
,
$\frac{{\partial e}}{{\partial{z}}}$
,
$\frac{{\partial e}}{{\partial l}}$
,
$\frac{{\partial e}}{{\partial s}}$
,
$\frac{{\partial e}}{{\partial k}}$
, and
$\frac{{\partial e}}{{\partial r}}$
can be obtained from Eq. 9 to Eq. 15.
\begin{equation} \begin{array}{l}{e_0} ={\left ({x - x_0} \right )^2} +{\left ({y - y_0} \right )^2} +{\left ({z - z_0} \right )^2} - \\{\left [{{l^0}\left ({x - x_0} \right ) +{s^0}\left ({y - y_0} \right ) +{k^0}\left ({z - z_0} \right )} \right ]}^2 -{r_0}^2 \end{array} \end{equation}
\begin{equation} \begin{array}{l} \frac{{\partial e}}{{\partial{x}}} = -2 *{l} * \left ({{l} * \left ({x - x_0} \right ) +{s} * \left ({y - y_0} \right ) +{k} * \left ({z - z_0} \right )} \right ) + 2 * \left ({x - x_0} \right ) \end{array} \end{equation}
\begin{equation} \begin{array}{l} \frac{{\partial e}}{{\partial{y}}} = -2 *{s} * \left ({{l} * \left ({x - x_0} \right ) +{s} * \left ({y - y_0} \right ) +{k} * \left ({z - z_0} \right )} \right ) + 2 * \left ({y - y_0} \right ) \end{array} \end{equation}
\begin{equation} \begin{array}{l} \frac{{\partial e}}{{\partial{z}}} = -2 *{k} * \left ({{l} * \left ({x - x_0} \right ) +{s} * \left ({y - y_0} \right ) +{k} * \left ({z - z_0} \right )} \right ) + 2 * \left ({z - z_0} \right ) \end{array} \end{equation}
\begin{equation} \begin{array}{l} \frac{{\partial e}}{{\partial l}} = - 2 * \left ({{l} * \left ({x - x_0} \right ) +{s} * \left ({y - y_0} \right ) +{k} * \left ({z - z_0} \right )} \right )* \left ({x - x_0} \right ) \end{array} \end{equation}
\begin{equation} \begin{array}{l} \frac{{\partial e}}{{\partial s}} = - 2 * \left ({{l} * \left ({x - x_0} \right ) +{s} * \left ({y - y_0} \right ) +{k} * \left ({z - z_0} \right )} \right )* \left ({y - y_0} \right ) \end{array} \end{equation}
\begin{equation} \begin{array}{l} \frac{{\partial e}}{{\partial k}} = - 2 * \left ({{l} * \left ({x - x_0} \right ) +{s} * \left ({y - y_0} \right ) +{k} * \left ({z - z_0} \right )} \right )* \left ({z - z_0} \right ) \end{array} \end{equation}
\begin{equation} \frac{{\partial e}}{{\partial r}} = - 2*{r} \end{equation}
Writing the error equation in matrix form yields Eq. 16:
\begin{equation}{N_{n \times 1}} ={A_{n \times 7}}{X_{7 \times 1}} -{O_{n \times 1}} \end{equation}
where
$A = \left [{\begin{array}{*{20}{c}}{\frac{{\partial e}}{{\partial{x_1}}}}&{\frac{{\partial e}}{{\partial{y_1}}}}&{\frac{{\partial e}}{{\partial{z_1}}}}&{\frac{{\partial e}}{{\partial{l_1}}}}&{\frac{{\partial e}}{{\partial{s_1}}}}&{\frac{{\partial e}}{{\partial{k_1}}}}&{\frac{{\partial e}}{{\partial{r_1}}}}\\ \cdots & \cdots & \cdots & \cdots & \cdots & \cdots & \cdots \\{\frac{{\partial e}}{{\partial{x_n}}}}&{\frac{{\partial e}}{{\partial{y_n}}}}&{\frac{{\partial e}}{{\partial{z_n}}}}&{\frac{{\partial e}}{{\partial{l_n}}}}&{\frac{{\partial e}}{{\partial{s_n}}}}&{\frac{{\partial e}}{{\partial{k_n}}}}&{\frac{{\partial e}}{{\partial{r_n}}}} \end{array}} \right ]$
and
$X ={\left [{\begin{array}{*{20}{c}} x&y&z&l&s&k&r \end{array}} \right ]^T}$
. Setting
${O_i} ={\left ({{x_i} -{x_0}} \right )^2} +{\left ({{y_i} -{y_0}} \right )^2} +{\left ({{z_i} -{z_0}} \right )^2} - r_0^2 -{\left [{{l^0}\left ({{x_i} -{x_0}} \right ) +{s^0}\left ({{y_i} -{y_0}} \right ) +{k^0}\left ({{z_i} -{z_0}} \right )} \right ]}^2$
, we can write
$O = \left [{\begin{array}{*{20}{c}}{{O_1}} &\cdots &{{O_n}} \end{array}} \right ]^T$
.
Finally, according to the least squares principle
${N^T}N = \min$
, the final cylinder parameters can be obtained from Eq. 17.
\begin{equation} X ={\left ({{A^T}A} \right )^{ - 1}}A{}^TO \end{equation}
Because some outliers will be filtered out when a point cloud is fitted to a cylinder, this method can also somewhat mitigate the influence of noise on the point cloud registration accuracy. The ICP algorithm is improved by introducing the above process of fitting each point cloud to a cylinder and adding a constraint between the fitted cylinder axis vectors. The calculation equations of the improved ICP algorithm are shown in Eq. 18.
\begin{equation} \left \{{\begin{array}{*{20}{c}}{F\left ({R,t} \right ) = \sum \limits _{i = 1}^n{||R{p_i} + t -{q_i}|{|^2}} }\\[5pt] {\arccos \left ({\frac{{\left ({{l_i},{s_i},{k_i}} \right ) \cdot \left ({{l_j},{s_j},{k_j}} \right )}}{{\sqrt{l_i^2 + s_i^2 + k_i^2} \cdot \sqrt{l_j^2 + s_j^2 + k_j^2} }}} \right ) \lt{\theta _{threshold}}} \end{array}} \right . \end{equation}
Here,
$\left ({{l_i},{s_i},{k_i}} \right )$
and
$\left ({{l_j},{s_j},{k_j}} \right )$
correspond to the cylinder axis vectors fitted from the two frame point clouds to be registered.
$\theta _{threshold}$
is a predefined angle threshold, and actual test results show that good results can be achieved when
${0.01^ \circ } \le{\theta _{threshold}} \le{1^ \circ }$
.
The above Eq. 18 can be solved using various optimization methods, such as the gradient descent method, the Gauss–Newton method, or the Levenberg–Marquardt algorithm. The main steps of the improved ICP-based point cloud registration algorithm are summarized in Fig. 2.
Figure 2. Flow chart of the improved ICP-based point cloud registration algorithm based on fitting with cylinder axis vector constraints.
4. Experimental results
To achieve intelligent operation and maintenance of steel pressure pipes for water diversion at large hydropower stations, the current mainstream solution worldwide is to use robots in place of humans to complete related inspection and maintenance tasks. Due to the inability of conventional robots to operate on vertical surfaces, customized robot designs are required for special scenarios such as hydropower stations. Based on the actual requirements of the inspection and maintenance tasks for the steel pressure pipes for water diversion at the Three Gorges Hydropower Station, a dual-track wall-climbing robot system was designed in this study, as shown in Fig. 3. The entire system mainly consists of an adsorption mechanism, a track module, a machine vision module, and a 3D reconstruction system. The technical parameters of the system are shown in Table I, and the subsequently reported experiments were conducted using the designed system.
Figure 3. Schematic diagram of the structure of the dual-track wall-climbing robot system.
Table I. Technical specifications of the dual-track wall-climbing robot.
To verify the effectiveness of the point cloud registration algorithm proposed in this paper, the diversion penstock at the Three Gorges Hydropower Station was taken as the reconstruction target in experiments. A structural diagram of the diversion penstock is shown in Fig. 4. The basic dimensions of the diversion penstock are 12.4 m in diameter and 110 m in axial length, and its shape is similar to that of a cylindrical pipe. The basic configuration of the experimental equipment is shown in Fig. 5, and the main parameters of the three-dimensional lidar data are shown in Table II. During the experiments, the wall-climbing robot was controlled by a remote controller to move along the axis of the diversion penstock. Due to the limited cable length, the wall-climbing robot could not collect laser point cloud data along the entire length of the pressure piping during its motion. Therefore, we selected only a section of the pressure piping for laser point cloud data collection and testing. The measured three-dimensional lidar data were collected by running the Robot Operating System (ROS) program on a Jetson Xavier NX controller.
Figure 4. Schematic diagram of the structure of the diversion penstock.
Figure 5. Actual three-dimensional reconstruction of the penstock and schematic diagram of the water flow reflecting the laser.
Table II. Main parameters of the 3D lidar.
First, to verify the feasibility of fitting point clouds to cylinders and obtaining the corresponding cylinder parameters, real point cloud data collected from pressure pipes were used, which contained a total of 8096 frames. The distribution of the cylinder radius values obtained via actual fitting is shown in Fig. 6.
Figure 6. Histogram of the radius parameter values obtained from cylindrical fitting of actual point cloud data collected from the diversion penstock.
As seen from the histogram of the fitted cylinder radius values, for 3825 frames of the fitted point cloud data, the cylinder radius distribution lies in the range of
$\left [{6.15,6.18} \right ]$
, which is very close to the true radius of the diversion penstock; thus, the histogram shows that the fitting results are correct. The cylindricity error of the fitted cylinders can be obtained from Eq. 19.
\begin{equation}{r_{RMSE}} = \sqrt{\frac{{\sum \limits _{i = 1}^n{{\left (r-{r_{truth}}\right )}^2} }}{n}} \end{equation}
Table III quantitatively compares the distributions of the fitted cylinder radii in different intervals, the percentage of the total number of point cloud frames that fall in each interval, and the root mean square error of the fitted cylinder radii.
Table III. Comparison of root mean square errors of the fitted cylinder radius values.
Figure 7 shows the distribution of the cylinder axis vectors obtained by fitting the actual laser point cloud data of the pressure pipes. The reference axis vector distribution represents the initial axis vector distribution, while the abnormal axis vector distribution represents the distribution of problematic axis vectors obtained when fitting the point clouds to cylinders. This figure indicates very good consistency of the fitted cylinder axis vectors, which is significant for the subsequent improvement of the ICP algorithm based on cylinder axis vector constraints. Figure 8 shows the results of a cylinder fitting test using the abovementioned point cloud fitting algorithm on real penstock point cloud data containing noise interference. As seen from this figure, cylinder fitting can eliminate the influence of the water flow on laser reflection in the point cloud.
Figure 7. Axial vector distribution map obtained from cylindrical fitting of actual three-dimensional lidar point cloud data collected from the diversion penstock.
Figure 8. Cylinder fitted from a real three-dimensional lidar point cloud of the diversion penstock.
Second, the same test data were used to quantitatively compare the accuracy of different point cloud registration algorithms. The test data were collected via lidar at different times at the same location. The original point cloud contained 60,057 data points. To improve the efficiency of point cloud registration, the original point cloud was resampled to a test point cloud with only 11,497 data points, and the initial position and attitude were set to 0. Table IV compares the results of different point cloud registration algorithms in terms of the determined position and attitude transformations and the running time. All algorithms except the improved ICP algorithm were implemented based on open-source code.
Table IV. Comparison of pose transformation parameters and running times of different point cloud registration algorithms.
All point cloud registration algorithms in Table IV were tested on the Jetson Xavier NX controller, which is shown in Fig. 9. The main technical parameters of the Jetson Xavier NX are given in Table V. Table IV shows that under the same conditions, the accuracy of the improved ICP algorithm is superior to that of other point cloud registration algorithms. However, although the improved ICP algorithm is more accurate than other ICP algorithms, its efficiency is reduced.
Figure 9. Schematic of the Jetson Xavier NX module.
Table V. Main technical parameters of the NVIDIA Xavier NX controller.
Based on the comparative testing of the point cloud registration algorithms mentioned above, the ICP, NDT, CPD, and improved ICP algorithms were further applied in consecutive frame-to-frame registration tests on the previously collected 8096 frames of penstock point cloud data. During the testing process, the parameters found for pose transformation between each pair of frames were recorded. Figure 10 and Fig. 11 show the position and attitude parameter change curves obtained by different frame-to-frame registration algorithms. It can be seen from these figures that the improved ICP algorithm results in gentler fluctuations in the position and attitude curves.
Figure 10. Comparison of the changes in the displacement parameters found by different interframe point cloud registration algorithms.
Figure 11. Comparison of the changes in the attitude parameters found by different interframe point cloud registration algorithms.
Then, for quantitative comparison of the interframe registration algorithms, three main indicators commonly used in point cloud registration, namely, the root mean square error, the maximum common point set, and the Hausdorff distance were calculated. The equation for calculating the Hausdorff distance is shown in Eq. 20. The quantitative comparison of the different interframe registration algorithms in terms of the calculated evaluation indicators is shown in Table VI.
\begin{equation} \left \{{\begin{array}{*{20}{c}}{H\left ({P,Q} \right ) = \max \left ({h\left ({P,Q} \right ),h\left ({Q,P} \right )} \right )}\\{h\left ({P,Q} \right ) = \mathop{\max }\limits _{p \in P} \{ \mathop{\min }\limits _{q \in Q} ||p - q||\} }\\{h\left ({Q,P} \right ) = \mathop{\max }\limits _{q \in Q} \{ \mathop{\min }\limits _{p \in P} ||q - p||\} } \end{array}} \right . \end{equation}
Here,
$H\left ({P,Q} \right )$
is the bidirectional Hausdorff distance between point sets p and Q, and
$h\left ({P,Q} \right )$
is the unidirectional Hausdorff distance from point set p to point set Q.
Table VI. Comparison of quantitative evaluation indicators for interframe point cloud registration.
Finally, by transforming the point cloud registration results obtained by different point cloud registration algorithms into a unified world coordinate system, laser-based odometry can be achieved. Due to the lack of higher-precision instruments for recording the actual movement trajectory of the lidar during the process of point cloud collection for the experimental penstock, relative evaluation metrics are used in this article to assess the accuracy of the various point cloud registration algorithms. Figure 12 and Fig. 13 show the position and attitude parameter change curves of laser-based odometry as implemented based on the different point cloud registration algorithms. It can be seen from these figures that the improved ICP algorithm yields the smallest cumulative error. A quantitative comparison is shown in Table VII.
Figure 12. Comparison of the position results obtained in lidar odometry based on different registration algorithms.
Figure 13. Comparison of the attitude results obtained in lidar odometry based on different registration algorithms.
Table VII. Quantitative comparison of the position and attitude parameters obtained in lidar odometry implemented based on different interframe registration algorithms.
Considering that extensive previous research on three-dimensional reconstruction algorithms has yielded several representative methods that achieve high reconstruction accuracy, such as LeGO-LOAM, FAST-LIO2, and DLO, this article first investigates the results obtained by using LeGO-LOAM, FAST-LIO2, and DLO directly for the three-dimensional reconstruction of the experimental penstock, without first fitting the point clouds to cylinders. These results are then compared with the three-dimensional penstock reconstruction results obtained after performing cylinder fitting of the point clouds to improve the point cloud registration in LeGO-LOAM, FAST-LIO2, and DLO, revealing the effectiveness of the proposed cylinder fitting approach in improving the accuracy of three-dimensional reconstruction.
LeGO-LOAM is based on feature extraction and realizes lidar odometry by means of the ICP algorithm. FAST-LIO2 directly registers the point clouds without feature extraction. In the actual registration process, inertial measurement sensor data are added to correct point cloud distortions. This method also relies on the ICP algorithm for point cloud registration. DLO is a lightweight front-end lidar odometry method that achieves accurate real-time pose estimation through generalized ICP point cloud matching.
Using the aforementioned three-dimensional reconstruction algorithms, practical three-dimensional reconstruction tests were conducted based on a portion of the penstock shown in Fig. 4. A comparison of the actual reconstruction effects before and after the incorporation of the proposed improvements into the three-dimensional reconstruction algorithms is shown in Fig. 14. It is clear from this figure that the improved reconstruction algorithms can achieve higher reconstruction accuracy.
Figure 14. Comparison of the three-dimensional reconstruction results obtained for the diversion penstock based on different reconstruction algorithms.
5. Discussion
This article has studied point cloud registration algorithms for the three-dimensional reconstruction of diversion penstocks at large-scale hydropower stations. Due to the sparse features and complex internal environmental conditions of diversion penstocks, existing three-dimensional reconstruction algorithms are often ineffective and unable to achieve accurate penstock reconstruction. By considering the geometric shape of a penstock as cylindrical, this article has proposed a method of fitting cylindrical pipes using point cloud data to obtain the axis vectors of a penstock. These axis vectors are then used as constraints in the traditional ICP algorithm, thereby improving the accuracy of three-dimensional reconstruction and overcoming uncertain environmental interference to some extent. However, the proposed improved ICP algorithm based on point cloud fitting with cylinder axis vector constraints still has some limitations because it is not applicable to target objects with noncylindrical shapes. Moreover, many different types of scenes with sparse features may be encountered in practical applications, some of which may not contain any regular geometric shapes, so the proposed method may not be generally applicable to scenes with sparse features. Therefore, further research is still needed to develop point cloud registration algorithms suitable for arbitrary scenes with sparse features.
6. Conclusion
This article mainly focuses on the registration of three-dimensional lidar point clouds for the three-dimensional reconstruction of scenes with sparse textures and features. Taking a diversion penstock with a certain geometric structure as an example, a geometric model of the point cloud data collected via three-dimensional lidar is fitted, and its parameters are obtained. Based on the obtained partial geometric model parameters, the widely used ICP algorithm is improved. Specifically, the axis vectors of the fitted geometric models of the point clouds to be registered are used to add constraints to the classical ICP algorithm, thereby constructing the proposed improved ICP algorithm. The feasibility of fitting point clouds to cylinders is verified based on actual point cloud data collected from a diversion penstock, and the same data are used to compare the improved ICP algorithm with current representative point cloud registration algorithms such as ICP, NDT, and CPD. Then, the pose parameters obtained through lidar odometry implemented based on different interframe registration algorithms are quantitatively compared. Finally, the improved ICP algorithm is integrated into three current open-source algorithms for three-dimensional reconstruction, and the reconstruction effects achieved in actual three-dimensional reconstruction of the diversion penstock using the resulting improved algorithms are compared with those of the original algorithms. The experimental results show that the accuracy of the improved ICP algorithm is higher than that of currently used schemes for both interframe point cloud registration and three-dimensional reconstruction.
Acknowledgments
We thank the Three Gorges Power Plant of China Yangtze River Power Co., Ltd., for providing project support, and we thank Green Harmonic Drive Technology Co., Ltd., for providing the experimental platform and technical support for the integrated motor.
Author contributions
Yulong Zhang conceived the original ideas, designed all the experiments, and subsequently drafted the manuscript. Enguang Guan and Yanzheng Zhao supervised the project and reviewed the manuscript. Baoyu Wang revised the format and charts of the manuscript. All authors have read and agreed to the published version of the manuscript.
Financial support
This work was supported by the National Natural Science Foundation of China under Grant 61806124.
Data availability statement
The data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
Competing interests
The authors declare no competing interests.
