2.1. Robotic pavement inspection systems

Inspection of pavement conditions has been receiving considerable interest for automation, marking the first stage of maintenance before the deployment of follow-up repair operations. Automated inspection deals with the flow of pavement condition data, from acquisition (perception), processing to decision-making. The majority of pavement condition data refers to surface geometric and dimensional data captured by vision-based (e.g., cameras) or optics-based (e.g., laser scanners) measuring tools (Yu and Salari, Reference Yu and Salari2011; Vittorio et al., Reference Vittorio, Rosolino, Teresa, Vittoria and Vincenzo2014; Jo and Ryu, Reference Jo and Ryu2015). The material and structure health condition data is usually collected by electromagnetic and ultrasonic devices (Behera et al., Reference Behera, Pradhan and Das2021; Rasol et al., Reference Rasol, Pais, Pérez-Gracia, Solla, Fernandes, Fontul and Assadollahi2022) and weather sensors also contribute to the data acquisition. Automated processing of the captured data is typically powered by data science or AI algorithms, including traditional algorithms (Yan and Zhang, Reference Yan and Zhang2021) and more recently smart neural networks such as CNNs (Fan et al., Reference Fan, Cao, Zeng, Li, Li and Wang2022; Li et al., Reference Li, Du and Jia2022a, Reference Li, Yin, Wang and Yan2022b; Sirhan et al., Reference Sirhan, Bekhor and Sidess2024) and object detection algorithms such as YOLOv5, YOLOv7, and YOLOv8 (Sholevar et al., Reference Sholevar, Golroo and Esfahani2022). Detecting and classifying the damage types and assessing their condition facilitates decisions around repair measures.

To embed automated data acquisition and processing capabilities into a robotic hardware system, special robotic pavement inspectors have been developed, examples of which are presented in Figure 1 in chronological order. These robots mainly appear to be able to inspect cracks and potholes. The captured geometric and dimensional data includes images (or multispectral images), videos and point cloud data, and material health condition data (e.g., concrete rebar and concrete corrosion), and also includes ground-penetrating radar (GPR) data and/or electrical resistivity data. Classical data processing algorithms include pattern recognition, random sampling and vote, and histogram analysis and local adaptive thresholding, while neural network-based inspection starts to appear in these robots from around 2018.

Figure 1.

Examples of developed pavement inspection robots over time in the literature.

In terms of the stated inspection performance in Figure 1, different assessment metrics have been applied for different purposes. For assessing the detection (classification) accuracy of individual surface defects such as cracks or potholes, a percentage statistically showing the rate of successful detection is usually provided: In the early work of Koch and Brilakis (Reference Koch and Brilakis2011), the defect recognition rate is 75%, while in later works after 2017 this value climbs up to over 97% (Phung et al., Reference Phung, Hoang, Dinh and Ha2017; Pan et al., Reference Pan, Zhang, Cervone and Yang2018; Sheta and Mokhtar, Reference Sheta and Mokhtar2022) due to the implementation of machine learning (e.g., neural networks). For assessing the inspection of surface unevenness, minimum detectable surface deviation values are used to show the inspection system’s capability (Yamada et al., Reference Yamada, Ito and Ohya2013). For benchmarking, managing, and comparing the pavement condition in a systematic way, traditional synthetic indicators from field tests (pavement condition index, etc.) can be mapped with the damage index given by pavement surface image processing to reflect the inspection accuracy (Mei et al., Reference Mei, Zampetti, Di Mascio, Fontinovo, Papa and D’Andrea2022).

Most robot inspectors in Figure 1 have been developed in ground (wheel)-based platforms while aerial (drone)-based robot inspectors can still play an important role in accessing remote locations. These robot prototypes have been initially validated for their inspection capabilities in the field. However, their deployment in real-life settings can be constrained by certain issues. The ground-based robots or so-called unmanned ground vehicles (UGVs) normally require road closures and large amounts of time for surface inspection through exhaustive exploration of the pavement. The drones or so-called unmanned aerial vehicles (UAVs) need to get permission from authorities before flying over sites. In addition, the battery endurance of these stand-alone machines is another concern.

2.2. Robotic pavement repair systems

Once the pavement distresses are detected and classified, repair plans need to be established and executed. Over the past decades, since the 1990s (Skibniewski and Hendrickson, Reference Skibniewski and Hendrickson1990), robotic pavement repair has gone through a development course where three ascending levels of automation have been or are being realized: low level—material handling, medium level—distress understanding and toolpath planning, and high level—autonomous navigation. Figure 2 shows some typical examples for these three levels, respectively.

Figure 2.

Examples of different levels of automation in pavement repair robot.

Material handling marks the first level of automation as it characterizes the nature of a pavement repair task which is originally labor-intensive. Four common material handling tasks in the repair process have been explored for automation, including material application (extrusion/spray), material removing (milling/cutting), lane markings painting, and object (e.g., slabs, cones) pick-and-place. Among these, sealant extrusion for crack or pothole repair purposes has been the main focus. Liu et al. (Reference Liu, Yang, Wang and Yam2022) provide a detailed overview of the automated crack sealing platforms since the 1990s where gantries (x/y/z frames) and robotic arms usually mounted on a moving vehicle are the two major manipulators for placing the sealant. Due to the stable structure and the ease of design and control, gantry-based platforms have been most widely used until now; however, the robotic arm-based platforms, emerging slightly later, can have wider working area, flexibility, and ease of folding.

As shown in Figure 2, the early development stage of automated sealant application features an extrusion process where the flowable sealant is generally pumped from a storage tank to the extrusion head held by the gantry or robotic arm. However, drivers need to manually drive the extrusion nozzle to follow cracks (e.g., input the crack locations through a user interface). University of California, Davis has developed a series of Longitudinal Crack Sealing Machines (Hargadon et al., Reference Hargadon, Olson and Woodall2006). All these crack-sealing vehicles in the early stage apply the conventional overband sealing method utilizing a fixed-width seal to enclose a crack while following its shape, which gives sufficient tolerance to the machine positioning error. For example, for the Operator Controlled Crack Sealing Machine (OCCSM) system, the overall offset error reaches within about 25 mm measured in the field (road surface) while applying an overband seal of 100 mm (Bennett et al., Reference Bennett, Feng and Velinsky2003). Through years of development, the latest automated material application capability carried by the commercialized Archway Roadmaster asphalt patching robot applies a material spray process for quick pothole repair (archwayroadmaster.co.uk); however, it still requires a qualified human worker to instruct the material placement operations.

Recent developments in large-scale extrusion-based three-dimensional (3D) printing technologies have accelerated the uptake of robotic sealant application (Jackson et al., Reference Jackson, Wojcik and Miodownik2018; Gong et al., Reference Gong, Cheng, Fang, Cheng, Liu and You2023). Although filling cracks or potholes does not technically refer to a 3D printing (form creation) process, it follows the same extrusion principle to place the sealant material into the designated cracks or potholes along a preset path (Schaefer et al., Reference Schaefer, Xu, Palin, Al‐Tabbaa and Iida2024). Some developed extrusion head prototypes for cementitious or bitumen sealants can be found in Awuah and Garcia-Hernández (Reference Awuah and Garcia-Hernández2022) and Cao et al. (Reference Cao, Yu, Cui and Li2022). The machine positioning accuracy (Brousek et al., Reference Brousek, Petr and Mendricky2022) and the flow rate control are both important to achieve a quality seal. Equipping the robotic arm with a track-based crawler extends its mobility for on-site sealing work (Gong et al., Reference Gong, Cheng, Fang, Cheng, Liu and You2023).

The second level of automation requires robotic pavement distress understanding and repair toolpath planning where robots start to show some extent of autonomy through on-board computation and processing. Despite being technically similar to the robotic inspection capability described above, distress understanding can happen after damages have been found (from pavement inspection) or as an integrated function for first-time pavement inspection. The purpose of distress understanding is to accurately calculate the detailed spatial, geometric, and dimensional information of a target defect (e.g., crack or pothole) using computer vision or other perception techniques (Tsai et al., Reference Tsai, Kaul and Yezzi2013). Based on the calculated geometry of the damage, robotic repair toolpaths can be automatically planned using optimization or prediction-based algorithms to achieve both rapidity and correctness of repair subject to the working life of repair materials (Guo et al., Reference Guo, Yu, Gong and Yi2017; Hunte, Reference Hunte2023).

Efforts in developing automated distress understanding using machine vision actually started early through automating the material handling process in the 1990s. An example is the University of California, Davis’ Automated Crack Sealing Machine with a crack recognition accuracy of 70% (Velinsky, Reference Velinsky1993b), although the University of Texas at Austin’s Automated Road Maintenance Machine implements a man–machine interaction strategy where a human operator routinely identifies cracks using captured pavement images (Haas et al., Reference Haas, Traver, Kim and Greer1997). Improvement in automated distress understanding can be seen later in the crack sealer from the Georgia Institute of Technology which uses LED light projection to aid the detection of cracks, resulting in an 83% accuracy (Holmes, Reference Holmes2010). The Automated Crack Sealer with Telescopic Manipulator marks the effective use of artificial neural networks in distress understanding with a detection precision of 99.425% (Yoo and Kim, Reference Yoo and Kim2016). Some smaller robots are developed afterward to improve the process autonomy, for instance, the ground robot from Rutgers University (Guo et al., Reference Guo, Yu, Gong and Yi2017) which utilizes a graph-based crack coverage (GCC) algorithm for automatic robot motion planning (with a mean squared error of the nozzle trajectory of around 19.8 mm), and the drone repairer from University of Leeds (Eskandari Torbaghan et al., Reference Eskandari Torbaghan, Kaddouh, Abdellatif, Metje, Liu, Jackson and Purnell2019) which automatically generates toolpaths from images as stated. The Robotic Crack Sealer from Robotic Maintenance Vehicles (RMV) marks the first/latest commercialized robotic system for automated distress understanding and repair toolpath planning, but it is still manually driven (rmv.llc).

The vision for robotic pavement repair is to establish an autonomous navigation capability leveraging simultaneous localization and mapping (SLAM) (Mur-Artal et al., Reference Mur-Artal, Montiel and Tardos2015; Mur-Artal and Tardós, Reference Mur-Artal and Tardós2017) and path planning technologies (Li et al., Reference Li, Du and Jia2022a, Reference Li, Yin, Wang and Yan2022b). This allows the robotic system to travel to designated sites on its own and conduct repair activities. Multimodal sensory fusion and embodied AI are needed for such autonomy. The ongoing EU-funded robotic HERON system (Katsamenis et al., Reference Katsamenis, Bimpas, Protopapadakis, Zafeiropoulos, Kalogeras, Doulamis and Lopez2022) and the ARRES robot from Robotiz3d (a spin-out company of the University of Liverpool) (University of Liverpool, 2020) under development appear to be examples for this level of automation.

2.3. The use of robotics in the maintenance of other road-related infrastructure assets

Apart from roads, robotics has also been applied to the maintenance of other related infrastructure assets, such as tunnels (Menendez et al., Reference Menendez, Victores, Montero, Martínez and Balaguer2018; Terato et al., Reference Terato, Yasui, Nitta and Masu2019) and bridges (La et al., Reference La, Lim, Basily, Gucunski, Yi, Maher and Parvardeh2013; Le et al., Reference Le, Gibb, Pham, La, Falk and Berendsen2017). However, these applications are essentially for inspection purposes rather than physical repair processes.

Montero et al. (Reference Montero, Victores, Martinez, Jardón and Balaguer2015) perform a comprehensive review on robotic tunnel inspection which summarizes eight robotic inspection methods including vision, mechanical impact, laser, drilling, cleaning, GPR, small robotics, and embedded sensors. The inspection target is the tunnel structure—walls rather than the pavement. The authors also introduce the partially autonomous robotic inspection system TunConstruct (González et al., Reference González, Martínez, Jardón and Balaguer2009) and an outlook into the fully automated inspection system ROBINSPECT (Loupos et al., Reference Loupos, Amditis, Stentoumis, Chrobocinski, Victores, Wietek and Lopez2014), both based on a wheeled moving vehicle.

Other than tunnel inspections, robotic systems for bridge inspection mainly target the bridge deck, that is, the pavement structure of the bridge road area. Similar ground-based mobile inspection robots with vision, GPR or electrical resistivity-based sensing capabilities have been developed to conduct inspection tasks (e.g., cracks) (La et al., Reference La, Lim, Basily, Gucunski, Yi, Maher and Parvardeh2013; Le et al., Reference Le, Gibb, Pham, La, Falk and Berendsen2017) with specific mapping and path planning methods (Lim et al., Reference Lim, La and Sheng2014). Other bridge elements under robotic inspection can include undersides, sides, piles (Sutter et al., Reference Sutter, Lelevé, Pham, Gouin, Jupille, Kuhn and Rémy2018), and suspension cables (Xu et al., Reference Xu, Dai, Jiang and Wang2021). In terms of physical treatment, Gucunski et al. (Reference Gucunski, Yi, Basily, Duong, Kim, Balaguru and Najm2015) have developed a mobile non-destructive rehabilitation robot for repairing the delamination inside the deck using drilling and grouting processes (Figure 3).

Figure 3.

Mobile manipulator-based non-destructive rehabilitation system for bridge deck repair (Gucunski et al., Reference Gucunski, Yi, Basily, Duong, Kim, Balaguru and Najm2015).

3.1. Sensing: real-time awareness of the pavement condition

As pavement distresses occur more rapidly worldwide, frequent inspection for new damage plays a significant role in preventing them from developing into potential hazards. Sending robotic inspectors to damaged sites under approval from authorities can increase the inspection efficiency compared to human inspection, but this generally requires awareness of damage locations beforehand, so that robots can be deployed to correct locations and they know where exactly to inspect. Relying on human road users to report damage as conventionally practiced, or even simply relying on the robotic inspectors in a limited number to exhaustively explore and spot damage before conducting precise inspection (measurement), are both inefficient and unrealistic, for example, due to battery life issues and likely requiring costly road closures.

A useful solution is to dramatically increase the inspection frequency by either bolting on low-cost and low-resolution vision-based inspection sensors onto older consumer vehicles (that do not have vision-based sensors) of road users (Anvo et al., Reference Anvo, Thuruthel, Taha, de Silva, Al-Tabbaa, Brilakis and Iida2023), or directly utilizing existing vision sensors (e.g., cameras) on modern consumer vehicles. This transforms the conventionally isolated ordinary consumer vehicles into connected autonomous vehicles (CAVs). The objective of such CAVs will be to collectively and instantly identify pavement distresses (or potential distresses) by numerous low-resolution sensing (imaging) attempts at every moment instead of carrying out precise and high-resolution inspection, and perform an initial triage (i.e., filtering out fake damage data) to allow effective data to be sent to a cloud-based road DT for further decision-making of repair actions, which can potentially create real-time awareness of pavement conditions.

Although the above method is promising, it is still a passive sensing pattern where apparent damage must have occurred to be able to be inspected. What is more preferred is a pro-active sensing pattern where a pavement condition is monitored through embedded sensors or even self-sensing materials (Hasni et al., Reference Hasni, Alavi, Chatti and Lajnef2017; Gupta et al., Reference Gupta, Lin, Lee, Buscheck, Wu, Lynch and Loh2021a, Reference Gupta, Savarese, Ganguli and Fei-Fei2021b) to predict the condition’s evolution and potential risk so that preventive measures can be taken in advance.

3.2. Control: machine autonomy

As reviewed in the previous section, state-of-the-art maintenance robots have been developed to achieve automated material handling and a certain extent of autonomy, such as on-board or human-guided distress understanding and toolpath planning using computer vision for low-level simple repair tasks. For more complex repair tasks, the data processing procedures are normally completed outside of the robot hardware system—a remote cloud-based centralized control hub, that is, a road DT. Such a configuration is capable of general processing of sensory data and decision-making as long as the AI model has been trained using a sufficient amount of data (e.g., from web images). It can have the potential problem of robustness where the maintenance robot will not be able to execute operations if the remote control hub is unexpectedly down. It may also require a certain amount of time for remote data processing and possibly be impacted by distance. A higher computational capability of the remote hub and better wireless remote communication (e.g., 5G for tele-operations; Moniruzzaman et al., Reference Moniruzzaman, Rassau, Chai and Islam2022) are required to achieve the desired robotic execution efficiency.

Going forward, higher autonomy in robotic control can be expected based on an on-board control mechanism. This is when the maintenance robot can process and interpret sensory data from the pavement and the surrounding environment, and plan and execute the repair toolpaths and robot motions to conduct unmanned repair treatments. This self-control capability can be established through SLAM, edge computing (Mao et al., Reference Mao, You, Zhang, Huang and Letaief2017), or embodied AI (Gupta et al., Reference Gupta, Lin, Lee, Buscheck, Wu, Lynch and Loh2021a, Reference Gupta, Savarese, Ganguli and Fei-Fei2021b) which has received much attention in robotics recently with the development of multimodal neural networks such as large language models (Chen et al., Reference Chen, Tworek, Jun, Yuan, de Oliveira Pinto, Kaplan, Edwards, Burda, Joseph, Brockman, Ray, Puri, Krueger, Petrov, Khlaaf, Sastry, Mishkin, Chan, Gray, Ryder, Pavlov, Power, Kaiser, Bavarian, Winter, Tillet, Petroski Such, Cummings, Plappert, Chantzis, Barnes, Herbert-Voss, Hebgen Guss, Nichol, Paino, Tezak, Tang, Babuschkin, Balaji, Jain, Saunders, Hesse, Carr, Leike, Achiam, Misra, Morikawa, Radford, Knight, Brundage, Murati, Mayer, Welinder, McGrew, Amodei, McCandlish, Sutskever and Zaremba2021). With embodied AI, a robot can learn and understand the complex dynamic environment through iterations of interactions with the environment without prior training and make corresponding behaviors on its own. Such a self-organized, decentralized control pattern without the need for a central control unit can provide a cheaper and more robust solution for the management and coordination of a large number of maintenance robots in dynamic road network.

3.3. Actuation: accurate and rapid repair implementation

Once maintenance robots are sent over to the sites, the target road areas are most likely temporarily closed for the repair process. On the one hand, the damaged surface needs to be repaired accurately to create a neat and smooth surface for efficient and safe transportation; on the other hand, it is also important to finish the repair task as soon as possible not only to reduce the impact of road closure on road users, but also to satisfy the setting times of the repair materials for fixing the defects.

The repair accuracy (quality) is firstly dependent on the material application manner (resolution). Taking crack sealing as an example, the conventional treatment applies an overband or inlaid (Standards for Highways, 2020) method which places a wide sealant band (low extrusion resolution) onto the crack, but this can ruin the pavement appearance and cause an increase in the surface slipperiness and thus the risk of accidents for road users (especially for motorists) (Murphy, Reference Murphy2016). Only injecting the required correct volume of sealant into the crack (high extrusion resolution) to leave no surface areas of exposed materials after the sealing appears to be a preferred solution (Schaefer et al., Reference Schaefer, Xu, Palin, Al‐Tabbaa and Iida2024). Fast simulations of material application such as through position-based fluids (Macklin and Müller, Reference Macklin and Müller2013) can help to verify the repair quality and inform robot actions. Other factors related to the accuracy of injection sealing can derive from several aspects including the sensing algorithms (computer vision, GPR, ultrasound, etc.) for correct perception and calculation of the defect geometries, the planned toolpath accuracy of the robot end-effector (relative to the defect geometries) and the mechanical positioning accuracy (e.g., joint accuracy) of the robot end-effector. Efforts need to be put into these aspects to establish the desired accuracy.

As for process efficiency, current practice indicates that suitable repair materials (regarding wet properties) and their application processes need to be selected for corresponding distress categories. For instance, for patching potholes or large-area shallows, a material spray process can be selected with viscous repair materials; yet for crack sealing, a material injection process with less viscous (more fluid) repair materials can be expected. When deploying robots for better process efficiency, the injection and/or spray head trajectory, robot trajectory, and the process parameters (e.g., travel speed, flow rate) need to be optimized for a single defect as well as for multiple defects on a road section using learning-based methods (Bayesian optimization, reinforcement learning, etc.) or evolutionary algorithms. Overall, the optimization of process efficiency needs to be considered together with repair accuracy to yield the best performance.

4.1. System composition and functionalities

To conduct maintenance operations autonomously, the AMP system is expected to have five modules (one for sensing, actuation, control, power supply, and mobility) following a similar architecture to those found in robotic mobile manipulator systems, presented in Table 1. A sketch of the AMP system is depicted in Figure 4.

Table 1.

Conceptual AMP system modules and functions

Figure 4.

A sketch of the conceptual AMP system.

The sensing module contains various high-resolution sensors (LiDAR, stereo camera, panoramic camera, GPR, UPV sensor, ER sensor, IR sensor, weather station, etc.) to collect comprehensive pavement condition data. The actuation module deals with the execution of repair treatments through a gantry, a material application kit (extruder, pump, storage tank, and air compressor), a mill head, a robotic arm, and a lane marking painter. The control module is responsible for planning and simulating extruder toolpath trajectories and vehicle trajectories and controlling robotic motions as well as conducting communications with the DT. The power supply module and the mobility module respectively provide the energy and structure supports to the maintenance activities.

4.2. Digital workflow

The way the AMP takes maintenance actions is based on comprehensive communications with the road DT using an established communication channel. Figure 5 shows the general digital workflow steps of the AMP with the ascending numbers signifying the sequence of activities taking place in an entire pavement maintenance cycle of the cyber-physical platform. An overview of all the sequential activities can be found in Figure A1 in the Appendix: Only the activities related to the AMP are explained in details below (linking to Figure 5), while others are related to the other two elements of the cyber-physical platform, that is, low-resolution sensing activities (No. 1–3) in the CAV and data processing/decision-making activities (No. 4,8,9,14) in the road DT.

1. 1) No. 5—Deploy AMP to collect “high-res” data: Once the road DT processes the “low-res” pavement distress data uploaded by the CAVs and confirm the action of collecting “high-res” data, it will send the instructions with preliminary repair information (defect location, expected repair time, expected repair material type, etc.) to the AMP. The data exchange hub at the AMP passes the DT’s instructions to the on-board computer which will interpret these instructions and check the current system status (in terms of power, repair materials, and actuation machinery, etc.) through interfacing with all the component status monitoring sensors. If the system is ready, the AMP’s data exchange hub will send a confirmation tag back to the DT and the robotic vehicle will travel to the designated site either on its own through self-navigation and autonomous driving, or driven by a human operator. If the system is not ready, the AMP will also notify the DT while preparing its system (e.g., charging or loading materials) before it can set off.

2. 2) No. 6—AMP carries out “high-res” data collection: When reaching the site, the AMP will stop at a near distance from the target defect, initiate all the relevant “high-res” sensors listed in Table 1 and check and confirm their status to get ready for the survey. Then the AMP activates these sensors at the same time for the purpose of data fusion while roaming across the defect using a slower travel speed to conduct the high-resolution survey. This systematic survey collects various types of data including RGB images, point clouds, videos, sensor spatial data (GPS location, latitude, longitude, ellipsoidal altitude, projected coordinates), and timestamp, GPR/UPV/ER signals, temperature, and atmospheric data. The collected data is stored at the temporary memory in the data exchange hub of the AMP.

3. 3) No. 7—AMP sends relevant “high-res” data to DT: The captured “high-res” data is organized into multiple “packages” with each data package containing sensory data captured during the same period of time; the data exchange hub will then send the data packages successively to the DT. The size of the data package is dependent on the real-time data transmission speed and data processing capability of the DT to allow maximal efficiency. The data packaging and transmission can happen while the AMP is conducting the survey to save time. The uploaded data includes the pavement geometric data as well as the GPS location and timestamp for the DT to detect exact damage information (geometries, dimensions, etc.). The AMP can also report its current status information (on-board materials, machinery condition, power storage, etc.) to the DT for decision-making.

4. 4) No. 10—Instructions for repair or return sent back to AMP: Depending on the high-resolution data processing results in the DT, generally two types of decisions will be made by the DT:

   a) AMP repair decision—if the damage is verified and assessed/approved to be treated by the AMP right away, a high-level treatment (process) plan will be generated and translated into operation-level instructions and sent to the AMP; and

   b) AMP return decision—if the damage is a false positive, or if it needs substantial repair or resurface that is out of the capability of the AMP and requires human intervention with more resources, a return instruction will be sent to the AMP from the DT.

5. 5) No. 11—AMP carries out treatment: Based on the operation-level instructions from the DT, the on-board controller of the AMP generates motion-level programs (including gantry tool paths and vehicle paths) for conducting the repair actions. Fast simulations of these actions in the virtual world are likely to be conducted in either the on-board controller or the DT to verify the robotic programs (Schaefer et al., Reference Schaefer, Xu, Palin, Al‐Tabbaa and Iida2024). The repair actions can include (but not limited to) cone placement, brushing/air blowing the pavement surface, material injection and/or spray, drilling and milling, litter picking, and lane marking painting. The AMP will firstly place the cones around the pavement defect to separate it from running traffic and then take repair actions as planned.

6. 6) No. 12—AMP carries out post-treatment “high-res” data collection: Once the repair treatment is undertaken, the AMP will wait for a certain amount of time for the material to settle and then re-conduct the high-resolution survey on the areas of change (i.e., where treatment has been applied) using the same method as described in the No. 6 action.

7. 7) No. 13—AMP sends post-treatment high-res data to DT: The re-captured “high-res” data (geometric changes during the treatment, and their GPS locations and timestamps, also including the information on which defects were repaired and which were only marked) is again packaged and sent to the DT as per the method stated in the No. 7 action.

Figure 5.

Schematic of the digital workflow of the AMP.

The above AMP system needs to act within the whole cyber-physical platform framework where the road DT instructs the functioning of the AMP. This DT-enabled autonomous maintenance scheme distinguishes itself from the other ongoing standalone autonomous maintenance robot solutions such as the HERON robot (heron-h2020.eu) and the ARRES robot (robotiz3d.com) through the cyber-physical interactions for higher efficiency and effectiveness. It should be noted that such AMP design only provides a basic concept which possesses essential components for undertaking autonomous maintenance operations. The sensor configuration and material application processes can vary with different road environments as well as the wheel-based platform and power strategy. Optimizations and iterations of designs of the AMP can be expected in the future to enable the practical integration of various units and reflect the actual development work. The ultimate autonomy will eventually be pursued through not only centralized control but also decentralized control.