Efficient Pavement Crack Detection and Classification Using Custom YOLOv7 Model
Abstract
It is crucial to detect and classify pavement cracks as part of maintaining road safety. The inspection process for identifying and classifying cracks manually is tedious, time-consuming, and potentially dangerous for inspectors. As a result, an efficient automated approach for detecting road cracks is essential for this development. Numerous issues, such as variations in intensity, uneven data availability, the inefficacy of traditional approaches, and others, make it challenging to accomplish. This research has been carried out to contribute towards developing an efficient pavement crack detection and classification system. This study uses state of the art deep learning algorithm, customized YOLOv7 model. Data from two sources, RDD2022, a publicly available online dataset, and the second set of data gathered from the roads of Malaysia have been used in this investigation. In order to have balanced data for training, many image preprocessing techniques have been applied to the data, such as augmentations, scaling, blurring, etc. Experimental results demonstrate that the detection accuracy of the YOLOv7 model is significant, 92% on the RDD2022 dataset and 88% on our custom dataset. This study reports the outcomes of experiments conducted on both datasets. RDD2022 achieved a precision of 0.9523 and a recall of 0.9545. On the custom dataset, the resulting values for precision and recall were 0.93 and 0.9158, respectively. The results of this study were compared to those of other recent studies in the same field in order toestablish a benchmark. Results from the proposed system were more encouraging and surpassed the benchmarking ones.
Keywords
References
F. Ni, J. Zhang, and Z. Chen, “Pixel-level crack delineation in images with convolutional feature fusion,” Struct Control Health Monit, vol. 26, no. 1, p. e2286, Jan. 2019, doi: 10.1002/stc.2286.
V. P. Tran, T. S. Tran, H. J. Lee, K. D. Kim, J. Baek, and T. T. Nguyen, “One stage detector (RetinaNet)-based crack detection for asphalt pavements considering pavement distresses and surface objects,” J Civ Struct Health Monit, vol. 11, no. 1, pp. 205–222, Feb. 2021, doi: 10.1007/s13349-020-00447-8.
S.-T. Liong, Y. S. Gan, Y.-C. Huang, C.-A. Yuan, and H.-C. Chang, “Automatic Defect Segmentation on Leather with Deep Learning,” Mar. 2019.
C. V. Dung and L. D. Anh, “Autonomous concrete crack detection using deep fully convolutional neural network,”Autom Constr, vol. 99, pp. 52–58, Mar. 2019, doi: 10.1016/j.autcon.2018.11.028.
H. Oliveira and P. L. Correia, “Automatic Road Crack Detection and Characterization,” IEEE Transactions on Intelligent Transportation Systems, vol. 14, no. 1, pp. 155–168, Mar. 2013, doi: 10.1109/TITS.2012.2208630.
A. Ashraf, A. Sophian, A. A. Shafie, T. S. Gunawan, N. N. Ismail, and A. A. Bawono, “Detection of Road Cracks Using Convolutional Neural Networks and Threshold Segmentation,” Journal of Integrated and Advanced Engineering (JIAE), vol. 2, no. 2, pp. 123–134, Sep. 2022, doi: 10.51662/jiae.v2i2.82.
Naoki Tanaka and Kenji Uematsu, “A Crack Detection Method in Road Surface Images Using Morphology,” in MVA, 1998.
H. Oliveira and P. L. Correia, “Automatic road crack segmentation using entropy and image dynamic thresholding,” in 2009 17th European Signal Processing Conference, 2009, pp. 622–626.
H. Zhao, G. Qin, and X. Wang, “Improvement of canny algorithm based on pavement edge detection,” in 2010 3rd International Congress on Image and Signal Processing, Oct. 2010, pp. 964–967. doi: 10.1109/CISP.2010.5646923.
S. Chambon and J.-M. Moliard, “Automatic Road Pavement Assessment with Image Processing: Review and Comparison,” International Journal of Geophysics, vol. 2011, pp. 1–20, 2011, doi: 10.1155/2011/989354.
J. Tang and Y. Gu, “Automatic Crack Detection and Segmentation Using a Hybrid Algorithm for Road Distress Analysis,” in 2013 IEEE International Conference on Systems, Man, and Cybernetics, Oct. 2013, pp. 3026–3030. doi: 10.1109/SMC.2013.516.
H. Yong and Z. Chun-Xia, “A local binary pattern based methods for pavement crack detection,” Journal of Pattern Recognition Research, vol. 5, no. 1, pp. 140–147, 2010.
L. Zhang, F. Yang, Y. Daniel Zhang, and Y. J. Zhu, “Road crack detection using deep convolutional neural network,” in 2016 IEEE International Conference on Image Processing (ICIP), Sep. 2016, pp. 3708–3712. doi: 10.1109/ICIP.2016.7533052.
A. Ghimire, S. Thapa, A. K. Jha, A. Kumar, A. Kumar, and S. Adhikari, “AI and IoT Solutions for Tackling COVID-19 Pandemic,” in 2020 4th International Conference on Electronics, Communication and Aerospace Technology (ICECA), Nov. 2020, pp. 1083–1092. doi: 10.1109/ICECA49313.2020.9297454.
A. Ghimire, S. Thapa, A. K. Jha, S. Adhikari, and A. Kumar, “Accelerating Business Growth with Big Data and Artificial Intelligence,” in 2020 Fourth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and
Cloud) (I-SMAC), Oct. 2020, pp. 441–448. doi: 10.1109/I-SMAC49090.2020.9243318.
D. Ma, H. Fang, B. Xue, F. Wang, M. A. Msekh, and C. Ling Chan, “Intelligent Detection Model Based on a Fully Convolutional Neural Network for Pavement Cracks,” Computer Modeling in Engineering & Sciences, vol. 123, no. 3, pp. 1267–1291, 2020, doi: 10.32604/cmes.2020.09122.
Y. Shi, L. Cui, Z. Qi, F. Meng, and Z. Chen, “Automatic Road Crack Detection Using Random Structured Forests,” IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 12, pp. 3434–3445, Dec. 2016, doi: 10.1109/TITS.2016.2552248.
A. Zhang et al., “Automated Pixel-Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep-Learning Network,” Computer-Aided Civil and Infrastructure Engineering, vol. 32, no. 10, pp. 805–819, Oct. 2017, doi: 10.1111/mice.12297.
Y. Fei et al., “Pixel-Level Cracking Detection on 3D Asphalt Pavement Images Through Deep-Learning- Based CrackNet-V,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 1, pp. 273–284, Jan. 2020, doi: 10.1109/TITS.2019.2891167.
Q. Zou, Z. Zhang, Q. Li, X. Qi, Q. Wang, and S. Wang, “DeepCrack: Learning Hierarchical Convolutional Features for Crack Detection,” IEEE Transactions on Image Processing, vol. 28, no. 3, pp. 1498–1512, Mar. 2019, doi: 10.1109/TIP.2018.2878966.
X. Feng et al., “Pavement Crack Detection and Segmentation Method Based on Improved Deep Learning Fusion Model,” Math Probl Eng, vol. 2020, pp. 1–22, Dec. 2020, doi: 10.1155/2020/8515213.
L. Pauly, H. Peel, S. Luo, D. Hogg, and R. Fuentes, “Deeper Networks for Pavement Crack Detection,” Jul. 2017. doi: 10.22260/ISARC2017/0066.
C. Feng, M.-Y. Liu, C.-C. Kao, and T.-Y. Lee, “Deep Active Learning for Civil Infrastructure Defect Detection and Classification,” in Computing in Civil Engineering 2017, pp. 298–306. doi: 10.1061/9780784480823.036.
H. Maeda, Y. Sekimoto, T. Seto, T. Kashiyama, and H. Omata, “Road Damage Detection Using Deep Neural Networks with Images Captured Through a Smartphone,” Jan. 2018, doi: 10.1111/mice.12387.
F. Yang, L. Zhang, S. Yu, D. Prokhorov, X. Mei, and H. Ling, “Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 4, pp. 1525–1535, Apr. 2020, doi: 10.1109/TITS.2019.2910595.
P. Chun, T. Yamane, and Y. Tsuzuki, “Automatic Detection of Cracks in Asphalt Pavement Using Deep Learning to Overcome Weaknesses in Images and GIS Visualization,” Applied Sciences, vol. 11, no. 3, p. 892, Jan. 2021, doi: 10.3390/app11030892.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Commun ACM, vol. 60, no. 6, pp. 84–90, May 2017, doi: 10.1145/3065386.
H. Oliveira and P. L. Correia, “CrackIT & #x2014; An image processing toolbox for crack detection and characterization,” in 2014 IEEE International Conference on Image Processing (ICIP), Oct. 2014, pp. 798–802. doi: 10.1109/ICIP.2014.7025160.
S. Zhao and H. Ning, “Detection of pavement cracks based on convolutional neural network,” Sensors and Microsystems, vol. 36, no. 11, pp. 135–138, 2017.
B. Qian, Z. Tang, X. Shen, and J. Guo, “Pavement crack detection based on multi feature manifold learning and matrix decomposition,” Journal of Instrumentation, vol. 37, no. 7, pp. 1639–1646, 2016.
Z. Fan et al., “Automatic Crack Detection on Road Pavements Using Encoder-Decoder Architecture,” Materials, vol. 13, no. 13, p. 2960, Jul. 2020, doi: 10.3390/ma13132960.
D. Arya, H. Maeda, S. K. Ghosh, D. Toshniwal, and Y. Sekimoto, “RDD2022: A multi-national image dataset for automatic Road Damage Detection,” Sep. 2022.
C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” Jul. 2022.
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” Jun. 2015.
Z. Fan, Y. Wu, J. Lu, and W. Li, “Automatic Pavement Crack Detection Based on Structured Prediction with the Convolutional Neural Network,” Feb. 2018.
G. X. Hu, B. L. Hu, Z. Yang, L. Huang, and P. Li, “Pavement Crack Detection Method Based on Deep Learning Models,” Wirel Commun Mob Comput, vol. 2021, pp. 1–13, May 2021, doi: 10.1155/2021/5573590.
Refbacks
- There are currently no refbacks.
Indonesian Journal of Electrical Engineering and Informatics (IJEEI)
ISSN 2089-3272
This work is licensed under a Creative Commons Attribution 4.0 International License.