Visible & thermal imaging and deep learning based approach for automated robust detection of potholes to prioritize highway maintenance
| dc.contributor.author | Chen, Wei-Hsiang, author | |
| dc.contributor.author | Jia, Gaofeng, advisor | |
| dc.contributor.author | Guo, Yanlin, committee member | |
| dc.contributor.author | Chen, Haonan, committee member | |
| dc.date.accessioned | 2023-06-01T17:27:08Z | |
| dc.date.available | 2023-06-01T17:27:08Z | |
| dc.date.issued | 2023 | |
| dc.description.abstract | Potholes are a primary pavement distress that can compromise safety and cause expensive damage claims. Potholes are results of deterioration of pavements due to aging, weather and traffic overloads and are common problems across the U.S. Potholes are even more common in the Mountain Plains region due to the snow and freeze/thaw effect. Identifying and repairing potholes is one critical aspect of highway maintenance. Accurate, robust and fast detection of potholes is critical to enabling timely and cost-effective pavement maintenance. Recently, there has been growing interest and research in using machine learning techniques for pothole detection using different views of visible images. However, quality of potholes detection using only visible images may be significantly compromised due to poor lighting, weather conditions, low contract to surrounding pavement. On the other hand, thermal images are more robust to lighting and weather conditions. Although thermal images may lack the texture details of visible images, they can offer additional unique features compared to visible images, e.g., temperature difference between pothole and surrounding pavement, which can be potentially used for pothole detection. However, so far, the great potential and effectiveness of integrating both visible and thermal images as well as using fused images to enable accurate and robust pothole detection have not been investigated. This research aims to develop an automated deep learning based pothole detection and mapping tool for highway maintenance using the fusion of visible and thermal images. First, a unique and valuable database of geotagged and labeled trios of visible, thermal and fused images is established for training pothole detection algorithms. This is done through collecting pothole images using a low-cost FLIR ONE thermal camera connected to a smart phone. These data are used to train the machine learning algorithms for pothole detection. To establish an accurate pothole detection algorithm, we proposed and compared the performances of three machine learning algorithms, i.e., Anisotropic Diffusion Fusion (ADF) + Mask R-CNN, RTFNet, and RTFNet with Enhancement Parameters (EPs). These algorithms differ in how the visible and thermal images are fused and used for pothole detection. We achieved the best F1-score of 93.7% in the daytime scenario by the RTFNet method and 90.9% in the nighttime scenario by the RTFNet with EPs method. To best leverage the information from the thermal images, in the end we developed a Bright-Dark detector to determine the lighting conditions of candidate testing images, and then feed the images to the respective algorithms for pothole detection. For images with potholes detected, we also developed a mapping tool to map the location of the pothole using GPS information of the images. In the end, the trained overall algorithm is packaged as a tool with graphical user interface (GUI) to facilitate its adoption by highway maintenance team. As more images are collected, the overall algorithm can be continuously improved to further increase the pothole detection accuracy. | |
| dc.format.medium | born digital | |
| dc.format.medium | masters theses | |
| dc.identifier | Chen_colostate_0053N_17652.pdf | |
| dc.identifier.uri | https://hdl.handle.net/10217/236581 | |
| dc.language | English | |
| dc.language.iso | eng | |
| dc.publisher | Colorado State University. Libraries | |
| dc.relation.ispartof | 2020- | |
| dc.rights | Copyright and other restrictions may apply. User is responsible for compliance with all applicable laws. For information about copyright law, please see https://libguides.colostate.edu/copyright. | |
| dc.subject | pothole | |
| dc.subject | machine learning | |
| dc.title | Visible & thermal imaging and deep learning based approach for automated robust detection of potholes to prioritize highway maintenance | |
| dc.type | Text | |
| dcterms.rights.dpla | This Item is protected by copyright and/or related rights (https://rightsstatements.org/vocab/InC/1.0/). You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). | |
| thesis.degree.discipline | Civil and Environmental Engineering | |
| thesis.degree.grantor | Colorado State University | |
| thesis.degree.level | Masters | |
| thesis.degree.name | Master of Science (M.S.) |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- Chen_colostate_0053N_17652.pdf
- Size:
- 12.64 MB
- Format:
- Adobe Portable Document Format