3.1. Onsite inspections and data collection

The first phase of the proposed procedure consists of performing onsite inspections, aimed at collecting data. Two types of data need to be gathered: (a) photos of the structural elements to be analyzed in DOC-Crack; (b) photos of cracks from different views to enrich the overall dataset for the training.

Concerning the first type of data, once the structural element to investigate is identified, two simple approaches could be exploited: (a) use of a UAV; (b) a human survey with a high-resolution camera. When using UAVs, a specific protocol should be followed. In particular, once the absence of obstacles has been assessed, the UAV performs the flight following a series of reference points. The latter are designed to create a virtual fence area, which the UAV follows according to a linear route, parallel to the longitudinal development of the structural element. The distance from the UAV to the structural element is estimated according to the optical cone of the machine and its resolution, by ensuring an overlap between consecutive photos of at least 30% and ensuring the same angle (as suggested in Wang et al. (Reference Wang, Demartino, Narazaki, Monti and Spencer2023)). Similar performance could be achieved through a high-resolution camera managed by human surveyors, as indicated above. Nevertheless, although this second option may be cheaper than the first, it is strongly influenced by the presence of obstacles (e.g., vegetation), the photographer’s skill, and the height of the structural element being investigated (i.e., the deck and its components). A possible solution to the above issues is represented by the by-bridge platform.

Regarding the second type of data, a consistent dataset of defects on different structural elements must be collected, for example, (Cardellicchio et al., Reference Cardellicchio, Ruggieri, Nettis, Renò and Uva2023; Ruggieri et al., Reference Ruggieri, Cardellicchio, Nettis, Renò and Uva2023, Reference Ruggieri, Cardellicchio, Nettis, Renò and Uva2025), ensuring that the minimum requirements for the subsequent steps are met. In particular, the collected dataset should be characterized by (a) scale, (b) diversity, and (c) quality. Regarding scale, as reported by Cardellicchio et al. (Reference Cardellicchio, Ruggieri, Leggieri and Uva2022), a consistent size of dataset, characterized by several examples and typology of defects, should be collected. In addition, data balance should be ensured. Concerning diversity, the dataset should be characterized by enough intra- and inter-class variations, that is, each feature should be described by data accounting for variation of appearance, position, viewpoint, and pose, without discarding images with noise and occlusions. Finally, high-quality images are required, since low-resolution data can negatively influence the accuracy of the detection models.

3.2. Robust image stitching for global bridge view

Limitations of the sensing equipment, mainly in terms of field of view and resolution, imply that a single, comprehensive view containing the structural element to monitor is often not gathered in a single image. Therefore, providing a robust and reliable stitching procedure is a mandatory prerequisite for analyzing large-scale defects and assessing the overall structural health status. To this end, this work incorporates a well-assessed, feature-based stitching methodology, whose results are shown in Figure 2.

Figure 2.

An example of image stitching. First, key points are located using the Harris detector. Then, invariant descriptors are extracted and matched for each key point to identify overlaps between images. Finally, the panorama is obtained using a warping transformation.

The pipeline begins by detecting and describing keypoints by means of a Harris corner detector (Harris et al., Reference Harris and Stephens1988), and local invariant descriptors, such as Scale-Invariant Feature Transform (SIFT) (Lowe, Reference Lowe2004), which were computed to match features reliably between partially overlapping images. Once possible matching points are computed, the Random Sample Consensus (RANSAC) algorithm (Fischler and Bolles, Reference Fischler and Bolles1981) was used to estimate a homography matrix. The primary motivation for the selection of RANSAC is its robustness to outliers, which ensures that the estimated homography is based on geometric consensus, thus minimizing misalignments. Finally, the starting images were warped according to the homography matrix and blended to create a seamless overall view of the structural element.

3.3. Dataset and preparation for detection algorithms training

The data collected following the procedure highlighted in Section 3.1 were used to create two different datasets. The first dataset, the large-scale defect dataset, was used to train a deep neural network for the crack detection task, that is, localizing areas of interest within the images where a crack is likely to be found. The second dataset, the high-precision crack segmentation dataset, was then used to train another neural network to perform semantic segmentation, thereby identifying cracks within the areas of interest identified by the detection algorithm.

3.3.2. Dataset for high-precision crack segmentation

The dataset for training the segmentation network originated from a set of 450 images, specifically selected for their high-quality representation of crack defects. This dataset was labeled by domain experts by hand-tracing the full extent of visible cracks. Also in this case, the CVAT tool was used, and the results were exported in the Dataset Management Framework (Datumaro) format (Datumaro, 2022).

After labeling, a patch-based generation method was exploited to create the training samples for the pixel-wise classifier. Specifically, a square filter of size $ F $ was used to extract a patch at each pixel location; each of these patches was then labeled as either a crack or a noncrack sample, based on the fact that its central pixel belonged to a crack or not. These generated $ M\times N\times 450 $ data samples where $ M\times N $ was the size of each image. Thus, this approach addresses data scarcity by generating data directly from the source pixel rather than through image-processing transformations, thereby mitigating the risk of overfitting. However, this process implied a severe class imbalance, with the noncrack images vastly outnumbering the crack samples. As such, a hard-negative mining sampling strategy was implemented to balance the data and improve performance.

Still, before generating the patches, the 450 images were randomly split into two distinct sets, namely a training and a validation set, using a 70−30 split. This was specifically designed to prevent data leakage and thereby improve the overall generalization capabilities of models trained on the dataset. Overall, this resulted in three datasets of roughly the same size, as detailed in Table 1.

Table 1.

Datasets created for high-precision crack segmentation

Table 1. Long description

The table consists of three columns and four rows including the header.

* Header Row: Image patch size, Number of crack patches, and Number of noncrack patches.

* Row 1: Image patch size 64 times 64. Number of crack patches is 195.971. Number of noncrack patches is 198.656.

* Row 2: Image patch size 128 times 128. Number of crack patches is 195.813. Number of noncrack patches is 198.656.

* Row 3: Image patch size 256 times 256. Number of crack patches is 194.376. Number of noncrack patches is 198.656.

A note below the table explains that noncrack patches are negatively undersampled for balance, and larger patch sizes yield slightly fewer crack samples because no padding was applied to border patches.

Note. The noncrack patches are negatively undersampled to provide a balanced dataset. The number of crack patches varies with the size of the considered patch; as expected, larger patches yield slightly fewer data samples. No padding was applied as border patches were not considered.

3.4. Crack detection and counting

The enabling step in the processing pipeline was provided by the object detection model, which was specifically trained to identify multiple defects within imagery representing structural elements of RC bridges. To this end, starting from the work in Ruggieri et al. (Reference Ruggieri, Cardellicchio, Nettis, Renò and Uva2025), two variants on the basic architecture of YOLO11 Jocher and Qiu (Reference Jocher and Qiu2024) were proposed. Specifically, the YOLO family was originally proposed by Redmon et al. (Reference Redmon, Divvala, Girshick and Farhadi2016) as a single-stage object detector, that is, a DL model capable of localizing and classifying objects of interest within an image using a single CNN. In Ruggieri et al. (Reference Ruggieri, Cardellicchio, Nettis, Renò and Uva2025), the concept of Convolutional Block Attention Module (CBAM) was applied to the neck of YOLO11, highlighting improved detection capabilities for the network. While improving the overall performance of the network, the use of CBAM poses some constraints; specifically:

• • Feature Interaction Limitations: CBAM decouples attention into the channel (i.e., monodimensional) and spatial (i.e., bidimensional) components. While effective, this approach prevents the capture of true 3D attention weights, which are given by relationships between the spatial dimensions and the channels. Consequently, distinctive features restricted to specific spatial locations or feature channels may be suppressed due to the global pooling performed by the CBAM attention, potentially leading to information loss.

• • Reliance on Heuristics and Parameters: The CBAM module was designed around a set of heuristics, such as specific kernel sizes and reduction ratios in its multilayer perceptron (MLP), thus introducing additional hyperparameters and computational overhead, increasing the complexity of model tuning and inference.

To address these issues, two attention mechanisms were separately explored. First, the Global Attention Mechanism (GAM) introduced in Liu et al. (Reference Liu, Shao and Hoffmann2021) was considered to address the limitation of feature refinement by focusing on retaining information across dimensions to strengthen interactions. The main innovation behind the GAM lies in its design, which minimizes information reduction while maximizing cross-dimensional feature interaction to enhance global interactive representation. GAM achieves this by utilizing sequential submodules, notably in its Channel Attention submodule, where the input tensor is permuted and then processed by an encoder–decoder MLP to learn channel-spatial dependencies.

The second explored mechanism was SimAM, introduced by Yang et al. (Reference Yang, Zhang, Li and Xie2021), which leverages a fundamentally different approach inspired by neuroscience principles, allowing for the calculation of full 3D attention weights in an efficient, parameter-free manner. Specifically, the SimAM mechanism is derived from the neuroscientific theory of spatial suppression, which states that the most informative neurons are those that display distinctive firing patterns within a neighborhood and, therefore, should be prioritized. To this end, SimAM computes the importance of a single target neuron n by defining an energy function $ {e}_n $ that measures the linear separability between n and its neighbors. A lower energy value indicates that the neuron is more distinctive and, therefore, more important for visual processing; consequently, the attention weight is computed as the inverse of the energy $ \sigma \left(1/{e}_n\right) $ .

Two specific sections of the YOLO11 original network were targeted by the insertion of these mechanisms, namely the backbone and the neck. As for the backbone, two reasons underlie this choice: the first is to provide a mechanism that forces the network to recalibrate feature weights early on during training, effectively dimming the background and non-relevant information while highlighting the foreground elements of the image. Another effect is a prioritization of the channels that respond to specific spatial frequencies, such as the edge of a crack, over generic texture channels. Regarding the neck, it is worth noting that this section of the network combines semantic features extracted at various abstraction levels, enabling multiscale detection. Inserting attention mechanisms in this part of the network helps in achieving two goals: first, when features from a deep layer are concatenated with those from a shallow layer, a semantic gap can arise; attention mechanisms can help in deciding which details from the shallow layer are actually relevant to the semantic representation provided in the deep layer. Second, since deep layers inherently lose spatial precision, inserting attention in the neck helps recover precise defect localization from the shallow layers.

The models were first trained from scratch on the detection dataset, split according to a 70−30 strategy. Consequently, the trained models could be used to localize cracks within the input imagery, allowing for the estimation of defect positions within the final stitched image. In addition, the use of the object detector enables the counting of the number of occurrences of the defect. To this end, a counting module was implemented to quantify all cracks across the stitched image, thereby evaluating the extent of the crack pattern.

3.5. Crack segmentation

Once the cracks were detected using the best-performing object detection model, a novel pixel-based approach was developed to estimate the pattern of the identified cracks. This approach was based on a specifically designed neural network, named DOC-CrackNet, aimed at framing the problem in a different and possibly simpler way. Specifically, instead of using classic segmentation approaches based on U-shaped networks, the idea was to frame the problem as a binary classification problem, thereby training a simple CNN to distinguish between areas whose center likely belongs to a crack and areas whose center does not. This allowed three main improvements over other existing approaches:

1. 1. The problem of data availability was solved, as it was possible to extract a large number of patches from a starting image. Ideally, given an image of $ M\times N $ pixels, the proposed approach could extract up to $ \left(M- fracW2\right)\times \left(N- fracW2\right) $ different patches, where $ W $ represents the size of the selected patch. This allowed an exponentially higher number of samples to populate the dataset for the investigation.

2. 2. The problem of interpretability was partially mitigated by the adoption of simpler models, such as the one used in this work, which was composed of only three stacked simple convolutional layers.

3. 3. Framing the problem as a binary classification allowed a probabilistic output that, in future works, can be exploited to further improve the overall results of the network by adopting a flexible, variable classification threshold.

Interestingly, it must also be noted that exploiting simpler models allows for the reduction of the overall computational weight of the network, potentially enabling the adoption of this approach even on constrained hardware in real-time. The feature map size-halving criteria established by (He et al., Reference He, Zhang, Ren and Sun2016) was followed to design DOC-CrackNet. The network scheme of DOC-CrackNet is shown in Figure 3.

Figure 3.

The architecture of the proposed network. Specifically, DOC-CrackNet accepts an RGB image of the desired size (in this case $ 64\times 64 $ ). The image is then fed to three consecutive convolution layers, after which it is reduced in size using a max pooling layer. Finally, a fully connected layer extracts a feature embedding vector of 576 elements, fed to a sigmoid classifier for the final decision.

Figure 3. Long description

The diagram illustrates a neural network pipeline consisting of five main stages from left to right.

* Input (R G B). Three overlapping colored squares (red, green, and blue) labeled 64 times 64 times 3. Solid lines connect a small kernel area to the next stage.

* Conv 1. A stack of blue feature maps labeled 62 times 64 times 64. Dotted lines indicate a spatial reduction or transformation process.

* Conv 2. A stack of blue feature maps labeled 60 times 32 times 32. Solid lines connect a kernel from this layer to the next.

* Conv 3 plus max pooling. A stack of blue feature maps labeled 58 times 16 times 16. A single line connects this stack to the next layer.

* Fully connected. A single vertical blue bar labeled 576. To its right is the sigmoid activation function equation rho of x equals 1 all over 1 plus e to the power of minus x.

A final arrow points from the equation to the word Decision at the far right.

The latest layer of DOC-CrackNet presents a sigmoid activation function, which was chosen over the softmax activation function as the problem is framed as a binary classification task. In other words, through the sigmoid activation function, DOC-CrackNet provides a probability value $ \sigma \in \left[0,1\right] $ representing the likelihood that the patch is a positive (i.e., it represents a crack) or a negative sample (i.e., it does not represent a crack). This evaluation requires fixing a threshold value $ \phi $ to distinguish between the two different cases and to provide the metrics used for the overall performance evaluation:

• • True Positives (TP): the number of patches whose central pixel represents a crack and is correctly classified as a positive sample.

• • True Negatives (TN): the number of patches whose central pixel does not represent a crack and is correctly classified as negative samples.

• • False Positives (FP): the number of patches whose central pixel does not represent a crack and is incorrectly classified as a positive sample.

• • False Negatives (FN): the number of patches whose central pixel represents a crack and is incorrectly classified as a negative sample.

The threshold is manually set using threshold-moving (Provost, Reference Provost2000) to maximize the value achieved by the F1 score, defined as:

(1) $$ F1=2\cdot \frac{P\cdot R}{P+r} $$

where $ P $ represents the precision, defined as:

(2) $$ P=\frac{TP}{TP+ FP} $$

while $ R $ is the recall, defined as:

(3) $$ R=\frac{TP}{TP+ FN} $$

Therefore, higher precision is achieved when the model can misclassify a few negative samples, while higher recall is achieved when the model can misclassify a few positive samples. The F1 score synthesizes these aspects within a single metric.

The outcome of this step is a binary estimation mask $ M $ , whose value is white for the points where the crack is identified and black otherwise. This allows for identifying and segmenting each crack within the original images, delineating its pattern within the structural element.

Finally, it is worth underlining that the structure of DOC-CrackNet was designed to be flexible enough to accept patches of different sizes as inputs, allowing the evaluation of the impact of contexts for different sizes on the overall performance of the network.

3.6. Estimation of main crack orientation

Assessing the principal orientation of a crack is a critical feature for structural health management. In other words, while cracks often exhibit complex, nonlinear geometries, providing a robust, first-order approximation of their dominant linear trend (e.g., horizontal, vertical, or diagonal) can provide important insights into the causes behind a certain defect, as well as future maintenance strategies that can be proven effective.

The orientation is estimated in a two-stage process. First, the binary segmentation mask $ M $ obtained after crack segmentation is preprocessed by applying a morphological erosion filter. This step filters out segmentation artifacts and spurious branches, therefore ensuring the analysis is performed on the main component of the crack.

Once the erosion filter was applied $ M $ , the resulting erosion mask $ {M}_{er} $ was used as an input to train a regressor by means of the RANSAC algorithm (Fischler and Bolles, Reference Fischler and Bolles1981). The motivation behind the use of RANSAC lies in the consideration that a simple linear regressor would estimate an overall trend that is extremely sensitive to outliers. Using a robust regressor would filter out residual segmentation noise, as well as the nonlinear portions of the crack, allowing for robust identification of the single best-fit line representing the dominant trend of the crack.

Once the optimal regression model $ {R}_i $ is computed using RANSAC, its coefficient is converted to an angle $ \alpha $ and used to evaluate the direction of the crack according to the thresholds shown in Table 2. As can be observed, horizontal cracks are classified as ranging from 0° to 25°, diagonal cracks are classified as ranging from 25° to 60°, and vertical cracks are classified as ranging from 60° to 90°.

Table 2.

Threshold for determining the orientation of the crack according to the slope of the estimated model R_i

Table 2. Long description

The table consists of three columns: Orientation, Minimum value (degrees), and Maximum value (degrees). It contains three data rows:

* Horizontal: Minimum value 0, Maximum value 25.

* Diagonal: Minimum value 25, Maximum value 60.

* Vertical: Minimum value 60, Maximum value 90.

A footnote specifies that while only values between 0 and 90 degrees are shown, thresholds can be adapted to all scenarios by applying an offset of plus or minus pi all over 2.

Note. For the sake of simplicity, only values between $ 0 $ and $ 90 $ degrees were considered. However, if an offset of plus or minus $ \frac{\pi }{2} $ is applied to the specified values, the thresholds can be adapted to all the possible scenarios.

It is worth specifying that the proposed approach for estimating crack orientation is mainly based on the classification provided by the Italian guidelines MIT (2021), which subdivides cracks into the three classes defined above. Obviously, more complex crack shapes could be experienced in some real-life cases, for which the classes subdivision like the one provided could be too simplistic. Nevertheless, during onsite inspections, the domain expert must follow the guidelines and rules and then needs to assume a class within the three classes above defined. Still, although DOC-Cracks allows to detect and to trace also complex cracks, the logic behind the tool (not only with reference to orientation, but in the near-complete definition of crack pattern) is to support domain experts, who should always validate the output provided by the proposed automatic tool. Finally, it is clear that more complex approaches could be employed to define orientation (e.g., instance-level skeletonization to map varying orientations along the continuous path of a single curved crack), which are beyond the scope of this paper and can be the object of future developments.

4.1. Training of object detector and automated crack segmentation

4.1.1. Full-scale RC bridge object detector

For the case at hand, the base dataset was gathered by following the procedure described in Cardellicchio et al. (Reference Cardellicchio, Ruggieri, Nettis, Renò and Uva2023); Ruggieri et al. (Reference Ruggieri, Cardellicchio, Nettis, Renò and Uva2023, Reference Ruggieri, Cardellicchio, Nettis, Renò and Uva2025). Specifically, a total of 10,779 images were acquired during surveys of the structural elements of existing RC bridges, mainly decks, piers, pier caps, and abutments. Most of the gathered images were related to one or more structural elements, showing several defects of different extents and intensities. According to the above-mentioned works, the first labeling was performed to identify seven types of defects related to RC bridges.

The results of testing different configurations of YOLO models with various attention mechanisms are detailed in Table 3. Crucial considerations can be made concerning how placing different attention mechanisms in different parts of the network influences its behavior.

Table 3.

Comparison of model performance across metrics

Table 3. Long description

The table compares performance metrics (Precision, Recall, F 1 Score, m A P 0.5, and m A P 0.5 to 0.95) for Y O L O v 11 variants (n, s, m, l, x) across six categories.

* Baselines: Y O L O v 11 x achieves the highest baseline F 1 Score at 61.98 percent.

* C B A M (Ruggieri et al. 2025): Y O L O 11 x leads this group with a Precision of 82.87 percent and F 1 Score of 64.36 percent.

* G A M (Backbone): Shows significant improvement. Y O L O 11 x reaches 91.35 percent Precision, 55.97 percent Recall, and 69.41 percent F 1 Score. Y O L O 11 l also shows high performance with 56.89 percent Recall and 62.02 percent m A P 0.5.

* G A M (Neck): Y O L O 11 x leads with 88.32 percent Precision and 68.05 percent F 1 Score.

* Sim A M (Backbone): Y O L O 11 m reaches 89.34 percent Precision, while Y O L O 11 x has an F 1 Score of 68.40 percent.

* Sim A M (Neck): High performance across the board. Y O L O 11 m achieves 90.87 percent Precision and 56.21 percent Recall. Y O L O 11 x reaches an F 1 Score of 68.75 percent and m A P 0.5 to 0.95 of 48.65 percent.

Bolded values indicate peak performance within specific categories, notably concentrated in the G A M Backbone and Sim A M Neck configurations for the larger model variants.

Note: Values in green indicate the best results, values in red the second-best, and values in orange the third-best.

Specifically, the integration of GAM within the backbone yielded an overall superior value for mAP 0.5:0.95 outperforming baseline architectures by approximately 2.4% when the largest model scale (YOLO11x) was considered. This implies that, by filtering background noise at the earliest stages of feature extraction, the model achieves higher robustness, resulting in a stronger, noise-free feature representation. This enhanced feature space enables the detection head to focus solely on structural defects, resulting in tighter, more effective, and precise bounding boxes. Another significant finding is that the insertion of SimAM within the neck resulted in the optimal F1 score for the medium-sized model, thus implying a better trade-off between precision and recall, with a top value of 69.46% . This therefore suggests that the 3D attention weights proposed by SimAM are effective during the multiscale fusion stage, allowing for reliable discrimination of true defects from complex textures.

It is worth framing the impact of the proposed architectural enhancements on different network sizes. Specifically, there is a larger contribution for smaller scales (i.e., nano and small), mainly with the parameters-free SimAM attention mechanism. This effectively compensated for the relatively limited capacity of such networks, showing relative improvements of nearly 15% when placed within the neck for YOLO11n. At the same time, the relative contribution of the attention mechanisms is less relevant (yet not negligible) for larger scales, mainly due to the high representation capability already embedded in those architectures. Consequently, a sweet spot can be determined for the medium-sized model, as demonstrated by the introduction of SimAM in the neck.

Overall, these findings suggest that using a combination of SimAM and GAM can provide further improvements and should be explored in future work.

4.1.2. Crack detection

The trained model was fine-tuned to focus on cracks on the second, smaller dataset. This was characterized by 450 images containing visible samples of cracks. For each image of this new subsample, a further annotation was performed by hand-tracing cracks. Some examples of the new labeling, along with their corresponding ground truth, are shown in Figure 4.

Figure 4.

On the right, the results of the labeling segmentation procedure on the image on the left are highlighted in light blue or pink.

According to the procedure defined in Section 3.4, crack patches were derived, and the resulting segmentation dataset was composed of 195.971 positive samples and 198.656 negative samples. Some of the results are shown in Figure 5, in which patches with and without cracks are reported.

Figure 5.

A sample of the segmentation dataset. On the first row, six patches where cracks were not found are shown, while on the second row, six patches where cracks were found are reported.

Figure 5. Long description

The top row contains six patches of uniform material surfaces without visible defects. These patches vary in color and texture, including dark brown, charcoal gray, black, light gray with minor pitting, medium gray, and tan.

The bottom row contains six patches exhibiting various types of structural cracks.

- The first patch on the left shows a faint, dark vertical hairline crack.

- The second patch features a prominent, jagged Y-shaped crack on a light gray surface.

- The third patch shows a deep, dark vertical fissure on a rough, yellowish-brown surface.

- The fourth and fifth patches are nearly identical, showing a wide, horizontal jagged crack with significant depth and shadow on a light, stony texture.

- The sixth patch on the far right shows a vertical crack along the edge of a tan-colored surface.

Subsequently, a data augmentation procedure was also considered, through a series of augmentation steps specifically tailored to retain the semantic meaning of each image. Specifically, random rotations around the central pixel, random flipping (both horizontal and vertical), and random adjustment in terms of sharpening, contrast, and saturation were considered. All the augmentation steps were designed to provide a larger variability of the data fed to the network while retaining the information provided by the central pixels of the patch, and then not varying its class.

As for the hyperparameters, binary cross-entropy was selected as the loss function, with SGD as the optimization algorithm and 25 epochs of training with a learning rate of 0.01. All these parameters were experimentally found to be optimal for the problem at hand. Finally, as for the size of the input patch, let us recall that, as already stated in Section 3.4, DOC-CrackNet was designed to be flexible enough to accept patches of different sizes as input. To test the influence of the contextual information at different scales, the tests were conducted using square masks of three different input sizes, that is, $ 65\times 65 $ , $ 129\times 129 $ , and $ 257\times 257 $ . These values were selected experimentally, considering the overall size of the images under investigation and the memory constraints of the hardware used during training. In the following, each version of DOC-CrackNet will be referred to using a suffix indicating the size of the input patch.

4.2. Optimal threshold and quantitative evaluation

To rigorously evaluate the framework and ensure there is no data leakage, the dataset was strictly partitioned at the image level prior to patch extraction. The performance of the segmentation module was quantified using Accuracy and F1 score. It is worth noting that standard full-mask spatial overlap metrics, such as the intersection over union (IoU) or the Dice coefficient, were not explicitly selected as evaluation criteria. This is due to the ultrathin morphology of structural cracks, which often span only a few pixels, and to the specific formulations of IoU and Dice coefficient in semantic segmentation, which make them highly sensitive even to single-pixel spatial shifts. In other words, a (structurally) valid crack prediction that is misaligned by a single pixel incurs an artificially severe penalty, rendering these metrics overly pessimistic for structural health monitoring. Therefore, the patch-based F1-score on the strictly separated dataset provides a more reliable representation of the framework’s detection capabilities.

Consequently, an extensive ablation study was conducted to determine the optimal patch size, select from three sizes ( $ 64\times \mathrm{64,128}\times \mathrm{128,256}\times 256 $ ), evaluate the impact of data augmentation, and determine the optimal classification threshold $ \sigma $ , which was selected in the range [0.1,0.9] with a discrete step of 0.1. The results in terms of Accuracy and F1 score are reported in Tables 4 and 5, respectively.

Table 4.

Accuracy of DOC-CrackNet for all the tested patch sizes and with or without augmentation

Table 4. Long description

The table consists of 11 columns and 7 rows. The first two columns are Size and Augm (Augmentation). The subsequent nine columns are numerical thresholds: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, and 0.9.

* Row 1: Size 64, Augm No. Accuracy values range from 78.97 at 0.1 to a peak of 88.43 at 0.8, ending at 88.01 at 0.9.

* Row 2: Size 64, Augm Yes. Accuracy values range from 80.44 at 0.1 to a peak of 90.18 at 0.5, then decreasing to 75.56 at 0.9.

* Row 3: Size 128, Augm No. Accuracy values range from 64.57 at 0.1 to a peak of 78.43 at 0.8, ending at 77.70 at 0.9.

* Row 4: Size 128, Augm Yes. Accuracy values range from 61.91 at 0.1 to a peak of 78.89 at 0.5, then decreasing sharply to 52.37 at 0.9.

* Row 5: Size 256, Augm No. Accuracy values range from 75.21 at 0.1 to a peak of 83.99 at 0.7, ending at 83.28 at 0.9.

* Row 6: Size 256, Augm Yes. Accuracy values are lower, starting at 51.07 at 0.1, peaking at 62.09 at 0.5, and ending at 49.75 at 0.9.

Table 5.

F1 Score of DOC-CrackNet for all the tested patch sizes and with or without augmentation

Table 5. Long description

The table consists of 11 columns. The first two columns are Size (64, 128, 256) and Augm (No or Yes). The subsequent nine columns are numerical thresholds from 0.1 to 0.9.

* Size 64, Augm No: Scores range from 82.57 at 0.1 to a peak of 88.85 at 0.7, ending at 88.00 at 0.9.

* Size 64, Augm Yes: Scores range from 83.57 at 0.1 to a peak of 90.35 at 0.4, then decreasing to 68.70 at 0.9.

* Size 128, Augm No: Scores range from 73.72 at 0.1 to a peak of 79.98 at 0.7, ending at 77.57 at 0.9.

* Size 128, Augm Yes: Scores range from 72.44 at 0.1 to a peak of 80.11 at 0.5, then dropping sharply to 12.22 at 0.9.

* Size 256, Augm No: Scores range from 79.86 at 0.1 to a peak of 84.90 at 0.6, ending at 83.09 at 0.9.

* Size 256, Augm Yes: Scores range from 67.61 at 0.1 to a peak of 69.87 at 0.4, then dropping significantly to 3.16 at 0.9.

According to the results in Tables 4 and 5, the architecture that significantly outperformed the others was DOC-CrackNet trained on 64×64 patches with data augmentation. If an optimal $ \sigma =0.5 $ was selected, this model achieved an accuracy of 90.18% and an F1-score of 90.28% . This performance discrepancy is directly related to the receptive field provided by each patch dimension. Because DOC-CrackNet classifies the central pixel based on the surrounding window, the 64×64 patch provides sufficient local morphological context to recognize the linear continuity of a crack without overwhelming the network with distant background texture. Conversely, the larger $ 128\times 128 $ and 256×256 patches introduce excessive global noise and severe class imbalance (i.e., a vastly higher ratio of background concrete pixels to thin crack pixels). This negatively impacts the classification of the central pixel, particularly when spatial augmentations are applied. Therefore, the 64×64 patch size with data augmentation and a threshold of $ \sigma =0.5 $ was selected as the optimal baseline for the integrated framework.

4.3. Orientation estimation

Once cracks were fully characterized, the procedure described in Section 3.6 was applied to the extracted patches for identifying their predominant direction. To this end, the effects of the erosion filter on three different masks are shown in Figure 6.

Figure 6.

Application of the erosion filter on three different masks. The general effect of applying the filter on the eroded masks caused an overall reduction of the noisy outliers.

Subsequently, Figure 7 reports the results in terms of orientation, showing a blue line indicating the inclination and a legend reporting the angle and the desired output (i.e., vertical, diagonal, and horizontal).

Figure 7.

The results of the orientation evaluation on the same masks are shown in Figure 6. As shown, the methodology could correctly evaluate the orientation of the crack in all three cases.

4.4. Qualitative evaluation and discussion

As the last step in assessing the effectiveness of DOC-Crack, an experimental evaluation is provided by performing a qualitative evaluation of the results achieved by DOC-CrackNet-129 on a set of images that do not belong to the training set. The images shown in Figure 8 were selected to highlight some of the main pros and cons of DOC-Crack by showing different situations. Specifically, Figure 8(a) shows the results that can be achieved when a single and visible crack is present in the image. In this case, DOC-CrackNet-129 provided robust and consistent results, despite the varying angles of the crack depicted in the patch. The same conclusions could be drawn by looking at Figure 8(c), where the network was able to identify an irregular and discontinuous crack with high reliability. However, it is worth noting that the network can be misled under specific circumstances due to certain occurrences in the provided patch. For example, Figure 8(b) shows some irregularities due to the formwork of the structural element, which in the patch appeared visually similar to vertical cracks. Analogously, Figure 8(d) presents several irregularities in the form of honeycombs and abrasions on the concrete surface, which could also be misinterpreted if the network is not provided with contextual information. These effects resulted in visual incoherence of the provided patches, as highlighted by the predictions provided by the network for both patches in Figure 8(b) and (d). In such cases, while the network is still able to properly track the main crack in both situations, the prediction was affected by a certain degree of noise. Future evolution of the algorithm should address these situations by adopting specific solutions, such as data preprocessing to filter out noisy backgrounds, fusion with 3D surface data to identify defects such as honeycombs, or semantic enrichment via knowledge embedded into large language models to provide contextual information and, therefore, filter out formwork effects on the structural elements.

Figure 8.

Qualitative analysis. Each labeled subfigure represents a different crack in the dataset (on the left), with the associated crack identified by CrackNet (on the right).

Figure 8. Long description

The figure contains four labeled sections, a through d, arranged in a two-by-two grid. Each section displays a side-by-side comparison of a grayscale or color photograph of a surface crack and a black-and-white binary mask representing the detected crack area.

* Panel a: Top-left. The photograph shows a wide, jagged horizontal crack in a dark gray concrete surface. The binary mask shows a thick, continuous white jagged line against a black background, accurately tracing the main crack path.

* Panel b: Top-right. The photograph shows a thin, faint crack on a light-colored wooden or textured concrete surface with vertical grain lines. The binary mask captures the main horizontal crack as a white line but also includes several vertical white streaks corresponding to the surface texture noise.

0* Panel c: Bottom-left. The photograph shows a fine, branching crack on a rough, tan-colored masonry surface. The binary mask shows a segmented white path that is thinner and more broken than in panel a, reflecting the finer nature of the crack.

* Panel d: Bottom-right. The photograph shows a very thin, horizontal crack on a light gray, weathered surface. The binary mask shows a central white horizontal path surrounded by significant white speckle noise, indicating a high level of surface interference in the detection process.