-
本文中提出的网络主要由三部分组成:(1)特征学习网络; (2)卷积层; (3)分类网络。如图 2所示。
将点云所在的3维空间分为大小完全相同的若干体素格。假定点云数据在3维空间内占据的长、宽、高分别为D, W, H; 且每个体素格的长、宽、高分别为vD, vW, vH; 对于体素格类别的判断分为两个步骤,首先令体素格的高度为H,对划分后的体素格进行第1次判断,而后对于已经确定为杆塔的体素格,假设垂直方向的体素格数量为n,令vH=H/n,在垂直方向上再次划分体素格,并进行进一步判断,这种做法避免了3维卷积运算,对整体的运算效率带来了较大提升,如图 3所示。
-
首先需要将点云数据中所有点归入对应的体素格中。由点云数据自身的特性可知,不同体素格中点的数量差距较为明显,基于运算效率以及稳定性的考虑,本文中算法对体素格内的点进行随机采样,设定阈值δn,当体素格内点的数量大于δn时,采用随机采样,只保留其中的δn个点; 当点的数量不大于δn时,则将所有点全部保留。
随机采样后,对于每个体素格,设其包含N个点,计算其中点的x, y和z坐标的均值xv, yv和zv:
$ \left\{ {\begin{array}{*{20}{l}} {{{\bar x}_{\rm{v}}} = \sum\limits_{i = 1}^N {\frac{{{x_i}}}{N}} }\\ {{{\bar y}_{\rm{v}}} = \sum\limits_{i = 1}^N {\frac{{{y_i}}}{N}} }\\ {{{\bar z}_{\rm{v}}} = \sum\limits_{i = 1}^N {\frac{{{z_i}}}{N}} } \end{array}} \right. $
(1) 对于体素格中每个点,可以从原始数据中得到Mi=[xi, yi, zi, ri]T,其中ri代表每个点对应的反射强度,在求得体素格中x, y和z坐标的均值之后,将Min=[xi, yi, zi, ri, xi-xv, yi-yv, zi-zv]T作为特征学习网络的输入,用于提取每个体素格中的形状、空间特征,如图 4所示。
其中前向传播部分由全连接层与线性整流函数(rectified linear unit, ReLU)组成。输入端Min经过特征层(feature layer, FL)FL0进行前向传播被投影为维度更高的特征矩阵,对于所有经由FL0产生的特征矩阵,按位置分别进行最大池化获得特征矩阵M00,继而将所有经由FL0产生的特征矩阵与M00相连接,得到每个点对应的特征矩阵L01,对于每一个L01,经过前向传播之后再次被投影为更高维度的特征L02,对其按位置分别进行最大池化,得到该体素格所对应的特征矩阵V0,同时利用最大池化的方式,也规避了3维点云的无序性对特征提取带来的影响。
-
卷积层由若干个卷积核组成,用于进一步提取各个体素格自身以及与周围其它体素格所共同形成的特征,为保证卷积后得到的特征数量与体素格的数量相等,需预先在特征图外围增加一圈空的特征向量,如图 5所示。
在此过程中引入传统的杆塔提取算法中的特征M′, 假设某体素格的特征向量V0是尺寸为nr×1的矩阵,则卷积过后得到的特征向量V1,其尺寸仍为nr×1,将V1与M′=[δz, q]T相连接,得到特征向量V2,其中δz代表体素格的高度特征值,q代表体素格内的点的数量(即点密度),如图所示,当体素格的高度取vH=H时,有:
$ {\delta _z} = {z_{\max }} - {z_{\min }} $
(2) 式中,zmax和zmin代表该体素格中所有点z的最大值与最小值,而当体素格的高度取H/n时,有:
$ {\delta _z} = \bar z - {z_{\min }}^\prime $
(3) 式中, z代表该体素格中所有点z的均值,而zmin′代表该体素格垂直方向上所有体素格中所有点z的最小值。结合δz和q两个特征,将得到的特征向量V2作为分类网络的输入以判断该体素格的类别。
-
如图 6所示,该部分以特征向量V2作为输入,以softmax层作为输出层Mout。
该分类网络包含全连接层与激活函数(ReLU),并以softmax模型作二分类的判断,经前向传播之后,将判断出该体素格的类型为杆塔点云或者非杆塔点云。其损失函数为:
$ {\mathit{\boldsymbol{E}}_{{\rm{loss }}}} = {\left( {{\mathit{\boldsymbol{M}}_{{\rm{label }}}} - {\mathit{\boldsymbol{M}}_{{\rm{out }}}}} \right)^2} $
(4) 式中,Eloss代表输出值与标定值的误差,Mlabel为事先标定的值,用于区别某体素格是否属于杆塔点云。当所有体素格的类型被判定完毕后,对其采用聚类算法,以获得每个杆塔的最终区域.
-
本文中的算法在Intel Corei5-7300HQ CPU,2.5GHz,8G内存硬件条件下运行,由于需要与以CloudCompare为基础的项目对接,因此在win 10,PCL 2.4.13.6环境下采用Microsoft Visual Studio 2015编程实现。
-
本文中所述网络采用北川地区的巡检数据作为训练集,其杆塔类型为V型塔或T型塔,在人工对杆塔点云位置进行标定之后,将其中80%作为训练集,20%作为测试集,并对原始数据进行平移、旋转、镜像等操作以达到数据扩展的目的。基于点云数据的特征考虑,如果体素格的尺寸过大,将影响杆塔提取的精度,而如果尺寸过小,造成每个体素格中容纳的点的数量减少,将难以提取到有效的特征,同样对杆塔提取造成不利影响。故体素格设置为如下尺寸(单位为m):vW=1m, vD=1m,而对于vH,在第1次判别时取vH=H,第2次进一步判别时取vH=2m, 并且第1次判别时体素格中点数上限为150,第2次判别时点数上限为30。
经由FL0产生的特征矩阵尺寸为32×1,与最大池化特征拼接后特征矩阵尺寸为64×1,经过第一部分特征学习网络,输出的体素格特征矩阵尺寸为128×1,第二部分卷积层中设置4个卷积核,第三部分分类网络中中间层的矩阵尺寸50×1,并采用mini-batch的方法进行训练。
由于训练集的单个点云数据文件包含的点数通常达到数百万以上,且其整体的长度和宽度可达1km甚至2km以上,考虑内存空间的问题,将原点云数据分块进行训练,每块的长度和宽度都为50m。故经过第一阶段特征学习网络之后,将得到一个尺寸为50×50且深度为128的特征。由于杆塔点云所占的体素格数目相对于点云整体的比例很小,若直接将数据输入进行训练,准确率的提升将较为迅速,而召回率则将一直维持在较低水平,故在训练中的反向传播部分对非杆塔点云的体素格进行欠采样,以维持召回率与准确率的平衡,如图 7所示。此为一块数据直接输入以及进行欠采样时,其召回率及准确率随训练次数的变化曲线,横轴为训练次数,纵轴为召回率或准确率的百分比。在训练数据集中,该网络的召回率与准确率均达到90%以上。
-
为验证本文中方法在不同环境下的可行性及鲁棒性,选取了不同地形以及包含不同塔型的3组机载激光点云数据进行实验。数据1覆盖面积为392m×1181m,包含3262599个数据点,地形较为平坦,周围有一定量树木干扰,其杆塔类型为酒杯型,数据2覆盖面积为173m×1165m,包含1573050个数据点,地形起伏较为明显,为杆塔定位带来一定困难,杆塔类型同样为酒杯型,数据3覆盖面积为771m×508m,包含1836981个数据点,高空中有若干噪声点,且档间有高大树木干扰,塔型为羊角塔与干字型塔,如图 8所示。经过聚类筛选之后,提取出杆塔点云,其总耗时分别为10s,12s,15s, 其中对体素格的聚类搜索半径为2m,聚类最小数量为10。
为验证杆塔点云的提取精度,对以上实验结果中总计9处杆塔的提取精度进行评估,评估标准为点云提取率(提取点数量/标定点数量)。
统计结果如表 1所示。由表 1可知,杆塔点云的提取精度通常能达到95%左右,无论是地物点云中存在地形起伏或者植被等因素的干扰,本文中提出的网络均能较好地提取杆塔点云,其中对平坦地形的提取效果最好,且运行速度也相对较快,而对地形明显起伏以及两档之间受高大树木干扰或者有噪声干扰的情况,其精度略有下降,但依然可以维持较高的准确率,说明本方法具有一定的鲁棒性。
Table 1. Accuracy evaluation of the extraction
data number of point
clouds calibratednumber of point
clouds extractedextraction
rate/%29554 29105 98.4 1 19306 18076 93.6 18898 18584 98.3 32930 30250 91.8 2 18822 17915 95.1 18086 17509 96.8 15941 15302 95.9 3 13079 12542 95.9 14291 13248 92.7
基于激光点云实现杆塔提取的轻量级网络
A lightweight network for power tower extraction from laser point cloud
-
摘要: 为了解决传统的杆塔点云提取算法对地面起伏较为敏感, 以及提取出的杆塔点云难以剔除地面点的问题, 提出了一种直接以原始点云数据为输入来实现杆塔提取的轻量级网络。将原始点云数据划分为若干大小相等的体素格, 利用特征学习网络及卷积网络提取体素格内的空间、结构特征; 并结合传统方法中的相对高度差以及点密度的特征, 从而判别该体素格为杆塔点云或非杆塔点云; 最后采用聚类算法剔除孤立的体素格以提高杆塔点云提取的准确率, 得到杆塔的激光点云数据。结果表明, 所提出的方法对于不同地形以及不同干扰因素情况下的杆塔, 提取精度能达到95%左右。该算法能有效地提取杆塔点云, 相对于格网法, 其稳定性及精确度有一定提升, 且对于高大树木、垂直遮挡等其它因素也有较好的抗干扰效果。Abstract: For the problem that the traditional algorithm for the extraction of power tower is sensitive to ground fluctuations, and some points of ground are hard to exclude from the extraction result, a lightweight neural network was proposed with a direct input of the original point cloud data to implement the extraction of power tower. By dividing the original point cloud data into a number of voxel grids of equal size, the feature learning network and a convolutional neural network were used to extract the spatial and structural information in the voxel grid. Then the characteristics of relative height and point density which can be found in traditional algorithms were added to the feature to determine whether the voxel is a type of tower point cloud or not. Finally, clustering was used to eliminate isolated voxels to improve the accuracy and obtain the laser point cloud data of power towers. The experimental results show that the algorithm has an accuracy of about 95% for different terrains and interferences in the extraction of power towers. The algorithm can effectively extract the point cloud of power towers, and has a certain improvement in stability and accuracy compared with the method using grid or other methods, and also has a good anti-interference effect on other factors such as the existence of tall trees or vertical occlusion.
-
Table 1. Accuracy evaluation of the extraction
data number of point
clouds calibratednumber of point
clouds extractedextraction
rate/%29554 29105 98.4 1 19306 18076 93.6 18898 18584 98.3 32930 30250 91.8 2 18822 17915 95.1 18086 17509 96.8 15941 15302 95.9 3 13079 12542 95.9 14291 13248 92.7 -
[1] ZHAO Y M, LI Y H, SHANG Y N, et al. Application and development direction of lidar[J]. Journal of Telemetry, Tracking and Command, 2014, 35(5): 4-22(in Chinese). [2] WU J J, LI L, FANG P K, et al. Effective organization and visualization of helicopter-based laser scanning data in power line inspection[J]. Laser Technology, 2019, 43(3): 318-323 (in Chinese). [3] WANG P, ZHOU Z, WANG S, et al. Investigating the working model of power line patrol based on airborne LiDAR[C]// 2016 IEEE PES 13th International Conference on Transmission & Distribution Construction, Operation & Live-Line Maintenance (ESMO). New York, USA: IEEE, 2016: 1-4. [4] WU J J, CHEN L, LI L, et al. Power line extraction and reconstruction from airborne LiDAR point cloud[J]. Laser Technology, 2019, 43(4): 500-505 (in Chinese). [5] GUO B, LI Q, HUANG X, et al. An improved method for power-line reconstruction from point cloud data[J]. Remote Sensing, 2016, 8(1): 36-52. doi: 10.3390/rs8010036 [6] SHI L, GUO T, PENG Ch, et al. Segmentation of laser point cloud and safety detection of power lines[J]. Laser Technology, 2019, 43(3): 341-346 (in Chinese). [7] LIN X G, ZHANG J X. 3-D power line reconstruction from airborne LiDAR point cloud of overhead electric power transmission corridors[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(3): 347-353. [8] WU X, YAN Z, WANG P, et al. A processing method for LiDAR data of power line patrol[C]//2016 4th International Conference on Applied Robotics for the Power Industry (CARPI). New York, USA: IEEE, 2016: 1-4. [9] LIN Y, WU D, LI D. High precisely surveying of power line patrol based on airborne LiDAR[J]. Urban Geotechnical Investigation & Surveying, 2011, 5(5): 71-74. [10] YOU A, WANG X, HAN X, et al. Applications of LiDAR in patrolling electric-power lines[C]//2013 The International Conference on Technological Advances in Electrical, Electronics and Computer Engineering (TAEECE). New York, USA: IEEE, 2013: 110-114. [11] APPLET P J, GOODFELLOW J W. Research on how trees cause interruptions-applications to vegetation management[C]//Rural Electric Power Conference, 2004. New York, USA: IEEE, 2004: 1307064. [12] POULIOT N, MONTAMBAULT S. Geometric design of the LineScout, a teleoperated robot for power line inspection and maintenance[C]//2008 IEEE International Conference on Robotics and Automation. New York, USA: IEEE, 2008: 3970-3977. [13] CHEN L. Information extraction of transmission line tower based on high density airborne laser point cloud[D]. Fuxin: Liaoning Technical University, 2016: 14-29(in Chinese). [14] SONG Sh. 3-D feature extraction of power transmission corridor based on airborne LiDAR point clouds[D]. Wuhan: Wuhan University, 2017: 6-23(in Chinese). [15] GUO T, SHEN P, SHI L. Study on power extraction and fast positioning from airborne LiDAR data[J]. Remote Sensing Technology and Application, 2018, 33(3): 530-535(in Chinese). [16] HAN W J. Three-dimensional power tower modeling with airborne LiDAR data[J]. Journal of Yangtze River Scientific Research Institute, 2012, 29(10): 122-126. [17] SHEN X J, QIN Ch, DU Y, et al. An automatic power line extraction method from airborne light detection and ranging point cloud in complex terrain[J]. Journal of Tongji University(Natural Science Edition), 2018, 46(7): 982-987 (in Chinese). [18] QI C R, SU H, MO K, et al. Pointnet: Deep learning on point sets for 3-D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE, 2017: 652-660. [19] QI C R, YI L, SU H, et al. Pointnet++: Deep hierarchical feature learning on point sets in a metric space[C]// Annual Conference on Neural Information Processing Systems 2017. Los Angeles, USA: Neural Information Processing Systems, 2017: 5099-5108. [20] ZHOU Y, TUZEL O. Voxelnet: End-to-end learning for point cloud based 3D object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE, 2018: 4490-4499.