1. Introduction
Design automation has been increasingly influenced by machine learning (ML) and data-driven approaches, with neural networks (NNs) being used for tasks such as automated analysis of 3D geometry and design-space exploration (Reference Bauer, Trapp, Stenger, Leppich, Kounev, Leznik, Chard and FosterBauer et al., 2024). These approaches can reduce manual effort and support faster iteration, particularly where products must be adapted to individual users or contexts. In such settings, learning-based methods can help recognise geometric patterns, identify functional features, or generate new design candidates, thereby supporting mass personalization. However, their performance strongly depends on high-quality training data, particularly 3D CAD models that reflect the structure, variability and constraints of real industrial products (Reference Fan, He, Liu, Song, Fan and YanFan et al., 2025).
Industrial CAD data introduces several challenges that complicate the development of reliable learning-based design automation. Unlike general-purpose 3D model repositories, including ModelNet (Reference Wu, Song, Khosla, Yu, Zhang, Tang and XiaoZhirong Wu et al., 2015) and S3DIS (Reference Armeni, Sener, Zamir, Jiang, Brilakis, Fischer and SavareseArmeni et al., 2016), industrial datasets are often proprietary, heterogeneous in structure and sparsely annotated (Reference Zhou, Tang and ZhouZhou et al., 2023). They often include complex design details, domain-specific features and parametric histories that are important for downstream tasks but rarely standardized across companies or product families. Preparing such data for NN training typically requires extensive preprocessing (cleaning and harmonising geometry and formats) and often manual labelling, while preserving the dataset’s real variability. For many engineering applications, collecting a sufficiently large and representative real-world dataset remains a main limitation for adopting ML-based approaches (Reference Yuksel, Eren and BörklüYuksel et al., 2025).
Synthetic datasets have therefore emerged as a practical way to address the scarcity of structured data. They can be created from parametric models (Reference Yuksel, Eren and BörklüYuksel et al., 2025), procedural rules (Reference Fan, He, Liu, Song, Fan and YanFan et al., 2025) or simulation-based generation (Reference Yuksel, Eren and BörklüYuksel et al., 2025) pipelines, and they enable controlled generation of large numbers of 3D models with controlled geometric variability, where geometric variability denotes dataset-level inter-model diversity (differences in overall shape across parts, not tolerance-level deviations from nominal geometry). Synthetic datasets can also provide complete labels such as feature identities, segmentation masks or parametric values at minimal cost. However, the usefulness of a synthetic dataset depends on how well it reflects geometry observed in industrial practice; otherwise, NNs trained on such data may fail when applied to real parts (Reference Fan, He, Liu, Song, Fan and YanFan et al., 2025; Reference Yuksel, Eren and BörklüYuksel et al., 2025).
To ensure representativeness, a real industrial dataset should first be systematically characterised. In this paper, geometric characterization refers to quantifying dataset-level geometric variability through consistent, distance-based comparisons between models, providing a practical reference for selecting, generating and/or augmenting synthetic datasets so that their variability better matches the one found in the industrial geometry of interest.
This study investigates the application of methods for characterizing real-world industrial 3D model collections to support the creation of more representative synthetic datasets. By analysing the geometric variability of a real industrial dataset, we identify indicative ranges of distance-based variability that synthetic datasets should aim to reproduce in order to better reflect real-world geometric diversity and improve their suitability for machine-learning applications. The overall goal is to provide a practical foundation for assessing and selecting representative subsets of synthetic datasets that more accurately capture industrial geometric variability, with the intention of supporting more informed transfer of models trained on synthetic data to real-world data.
2. Background
ML methods, and especially NN architectures, increasingly support different engineering tasks (Reference Mumuni, Mumuni and GerrarMumuni et al., 2024). The performance of ML and NN models used in tasks such big data analysis, classification, segmentation and the generation of new complex geometric data strongly depends on the choice of learning architecture and on the structure, variability and overall quality of the data used during training (Reference Bauer, Trapp, Stenger, Leppich, Kounev, Leznik, Chard and FosterBauer et al., 2024). Engineering data can take many forms, including numerical sequences, 2D images and different 3D representations. Depending on the data type, different strategies have been developed to prepare, augment and characterize datasets before they are used for neural network training (Reference Bauer, Trapp, Stenger, Leppich, Kounev, Leznik, Chard and FosterBauer et al., 2024). The topics covered in this section therefore focus on synthetic dataset generation, the importance of dataset quality, and existing approaches for characterizing 3D model collections. Together, these topics provide the conceptual and methodological context needed to position this study within the broader field of geometric dataset analysis for NN applications.
2.1. Synthetic 3D datasets in engineering ML applications
The main advantage of using synthetic datasets in the development and application of learning-based algorithms for engineering tasks is the ability to generate a large amount of data with controlled geometric variability. Such data can be automatically labelled, which significantly reduces the time costs and human errors that appear during manual annotation. Another advantage lies in the possibility of generating boundary or rare cases that are difficult or impossible to obtain from real industrial datasets. Synthetic datasets have therefore been applied in many different engineering fields, especially within generative design workflows and in tasks that require the analysis or processing of 3D geometry.
Several studies demonstrate how synthetic datasets can be implemented to support engineering ML tasks. In the field of manufacturing feature recognition, parametric CAD models were used to generate large synthetic datasets that served as training data for NNs. These datasets provided complete and reliable labels that would be difficult to extract from real CAD models (Reference Lenover, Bedi, Mann and MelekLenover et al., 2024). In the context of scene segmentation, synthetic datasets can be created by inserting specific 3D objects into virtual scenes. Reference Lopez Morales, Katsimpalis, Haas and NarasimhanLopez Morales et al. (2025) present a framework for preparing and incorporating 3D models into industrial scenes for semantic segmentation tasks. In engineering applications, especially in object classification and object detection, synthetic datasets often consist of large collections of 2D images rendered from 3D geometric models. Reference Wong, Kunii, Baylis, Ong, Kroupa and KollerWong et al. 2019 propose a pipeline for generating such datasets directly from physical or CAD representations of the target objects. Furthermore, synthetic datasets can be used for training generative NNs. For example, Reference Wu, Xiao and ZhengWu et al. (2021) introduce the DeepCAD model, which was trained on a large dataset of synthetically generated 3D models stored in a tailored CAD representation that captures both geometry and design intent.
Generation of synthetic datasets based on parametric CAD models enables systematic creation of geometric variants. However, only a small portion of parameters in a parametric model are truly independent, while many others are mutually constrained through mathematical relations (Reference Yuksel, Eren and BörklüYuksel et al., 2025). Because of these dependencies, changing one parameter often alters several other geometric characteristics of the model (Reference Fan, He, Liu, Song, Fan and YanFan et al., 2025). Therefore, parameter values must be varied in a controlled, systematic way to produce the required level of geometric diversity. Defining appropriate parameter ranges is often the most challenging step, because the real geometric variability of industrial components is usually unknown (Reference Zhou, Tang and ZhouZhou et al., 2023).
To address the challenge of parameter variation, several approaches can be used to ensure sufficient geometric diversity. Instead of relying solely on the full parametric structure, variation can be introduced by systematically sampling across the feasible design space defined by the model’s constraints. Even a small number of parameters can generate a large number of design variants when their values are explored across the full admissible range (Reference Yuksel, Eren and BörklüYuksel et al., 2025). This makes it possible to generate large families of models, although even a relatively small number of input parameters can produce a large number of different design variants. Parameter variability can also be introduced by randomly sampling combinations of input values, enabling additional coverage of the design space (Reference Sebestyen, Özdenizci, Hirschberg and LegensteinSebestyen et al., 2023).
To overcome the limitations of simple parameter changes and to ensure more complex geometric diversity, advanced geometric concepts such as Voronoi diagrams can be integrated into the parametric design process (Reference Yuksel, Eren and BörklüYuksel et al., 2025). Further geometric variability can be achieved through augmentation techniques, for example, by applying rotations, scaling operations, or small geometric perturbations to the generated models (Reference Yuksel, Eren and BörklüYuksel et al., 2025).
2.2. Geometric variability in 3D model datasets
Similarity metrics are fundamental tools for analysing geometric diversity within datasets. They enable the quantification of geometric variability between models, which is essential for the characterization of datasets. Choosing the appropriate metric is crucial, as the selection heavily depends on the type of 3D representation and the specific geometric characteristics of the models (Reference Dommaraju, Bujny, Menzel, Olhofer and DuddeckDommaraju et al., 2023). An ill-suited metric can lead to misleading results, making the analysis and comparison of models unreliable and ultimately rendering the dataset unfit for further use (Reference Alibekov, Staderini, Ramachandran, Schneider and AntensteinerAlibekov et al., 2024). Many metrics and their variations exist, especially within families such as Hausdorff distance, Chamfer distance, and point-to-surface distances. For the same 3D representation, several metrics can be used, but each metric highlights a different aspect of geometric variants (Reference Wu, Xiao and ZhengT. Wu et al., 2021).
Hausdorff distance is a measure commonly used for comparing two sets of points (point clouds), which captures the maximum deviation between two point sets and is therefore highly sensitive to outliers and noise (Reference Alibekov, Staderini, Ramachandran, Schneider and AntensteinerAlibekov et al., 2024). From this metric, several variants were developed, including the Gromov–Hausdorff distance, which provides additional insight into the topological properties of the compared shapes (Reference Adams, Frick, Majhi and McBrideAdams et al., 2025). On the other hand, Chamfer distance measures the minimal distances between two sets of points (Reference Wu, Xiao and ZhengT. Wu et al., 2021), while Earth Mover’s Distance provides a more robust comparison by capturing distribution shifts between point sets, but it comes with significantly higher computational cost (Reference Yang, Wang, Sun and PengYang et al., 2022).
Similar to the comparison of point sets, dedicated similarity metrics exist for mesh and voxel-grid representations (Reference Shin, Fowlkes and HoiemShin et al., 2018). For example, voxel models can be compared using the voxel Intersection-over-Union metric, which measures the volumetric overlap between two models (Reference Shin, Fowlkes and HoiemShin et al., 2018). Mesh models can be compared with other meshes using metrics such as mean surface distance, or with point clouds using cloud-to-mesh or point-to-surface distances (Reference Shin, Fowlkes and HoiemShin et al., 2018).
Similarity metrics can be used to compare any pair of 3D models (Reference Dommaraju, Bujny, Menzel, Olhofer and DuddeckDommaraju et al., 2023). When all models within a dataset are inter-compared, the result is a matrix that contains distance or similarity values for all pairs of models in the dataset (Reference Farzamnik, Ianiro, Discetti, Deng, Oberleithner, Noack and GuerreroFarzamnik et al., 2023). A matrix of this kind directly shows the most similar and most different pairs of models, as well as the overall range of similarity values present in the dataset (Reference Dommaraju, Bujny, Menzel, Olhofer and DuddeckDommaraju et al., 2023). To enable a more detailed characterisation of the entire dataset, methods such as manifold-based analytical techniques can be applied to the computed matrix.
In general, manifold-based analytical methods are used to analyse structure in high-dimensional spaces or other complex spaces such as distance or similarity spaces (Reference Farzamnik, Ianiro, Discetti, Deng, Oberleithner, Noack and GuerreroFarzamnik et al., 2023). These methods transform data from a complex space into a Euclidean 2D space without tearing the structure apart. Several analytical methods of this type exist, including Multidimensional Scaling (MDS) (Reference Radvar-Esfahlan and TahanRadvar-Esfahlan & Tahan, 2013), Isometric Feature Mapping (Isomap) (Reference Farzamnik, Ianiro, Discetti, Deng, Oberleithner, Noack and GuerreroFarzamnik et al., 2023), Principal Component Analysis (PCA) (Reference Dommaraju, Bujny, Menzel, Olhofer and DuddeckDommaraju et al., 2023), and Uniform Manifold Approximation and Projection (UMAP) (Reference Dommaraju, Bujny, Menzel, Olhofer and DuddeckDommaraju et al., 2023). The choice of method strongly depends on the specific goal of the analysis and the available computational resources (Reference Dommaraju, Bujny, Menzel, Olhofer and DuddeckDommaraju et al., 2023; Reference Radvar-Esfahlan and TahanRadvar-Esfahlan & Tahan, 2013). For example, MDS is one of the faster linear dimensionality reduction methods but preserves only pairwise distances. On the other hand, UMAP requires more computational effort but preserves both inter-cluster distances and local topology (Reference Radvar-Esfahlan and TahanRadvar-Esfahlan & Tahan, 2013).
2.3. Research question
Although synthetic datasets can be generated in large quantities and at low computational cost, training NN models on such large datasets requires significant time and computational resources. For this reason, only a small subset of the generated synthetic models is typically used for training (Reference Sebestyen, Özdenizci, Hirschberg and LegensteinSebestyen et al., 2023). However, if this subset does not adequately reflect the geometric variability of the real industrial components, the trained model may perform poorly on real-world data. This leads to the central research question of this study: How can geometric similarity metrics be used to guide generation of synthetic 3D model data that reflects real industrial variability without compromising NN performance?
3. Dataset characterization approach
The two-stage geometric characterization approach applied in this study consists of several sequential steps. It starts with normalizing the industrial and synthetically generated datasets in a unified and comparable form. After that, suitable geometric similarity metrics were applied to generate pairwise distance matrices for each of the datasets. These were then analysed in two stages. The first focused on the distribution of geometric distances, while the second included manifold-based embedding analysis. Together, these stages provide a structured characterization pipeline for examining geometric variability within any 3D model dataset.
3.1. Dataset preparation
Before applying any metrics for characterization of 3D models, real-world industry and synthetically generated datasets must be pre-processed in a consistent manner. Proper dataset preparation ensures that the observed differences between models originate from their geometric characteristics rather than from differences in representation, alignment, or scaling. This step establishes a common analytical foundation and enables a fair and reliable comparison between the real and synthetic datasets.
Firstly, the same type of 3D model representation (e.g. CAD models, voxel grid, point cloud, mesh model) must be used across all models. For example, if the models in the first dataset are stored as point clouds, while the second dataset contains models in STL format, then all models in the second dataset should be converted to point clouds.
Secondly, to eliminate the dependence of similarity metrics on model placement in 3D space, all models must be aligned. One of the simplest alignment methods is translating models so that their centroid is positioned at the origin of the coordinate system (or any other consistent reference point). Moreover, all models can be additionally aligned by applying rotational transformations to improve their mutual overlap and reduce the impact of differently rotated models on the values of the similarity metrics.
Finally, if the effect of model size needs to be eliminated, the model representations should be normalized. This can be performed using different techniques, where normalization to a unit cube or to a unit sphere are most often applied (Reference Adams, Frick, Majhi and McBrideAdams et al., 2025). However, if preserving the real dimensions is important for the dataset or for downstream tasks, this step can be omitted, although in that case metric values will also incorporate absolute size differences.
3.2. Calculation of similarity and distance metrics
The selection of a suitable similarity metrics depends primarily on the available 3D model representations. For a single representation, multiple metrics may be applied, and the choice between them depends on factors such as available computation time, computational efficiency, or the need to emphasise small geometric differences. Common options for 3D geometry comparison include Gromov–Hausdorff distance, Earth Mover Distance, point-to-face distance, and voxel Intersection over Union. In this study, we focus on Hausdorff distance, which measures the largest local deviation between two models by identifying the point furthest from any point on the other model. This metric is highly sensitive to outliners but provides a strict upper bound of geometric difference (Reference Adams, Frick, Majhi and McBrideAdams et al., 2025).
3.3. Computation and analysis of the pairwise similarity matrix
All previously mentioned metrics are applied to discrete sets of points, facets, or voxels, which introduces the problem of sampling the same number of elements from all models in the dataset. To ensure a fair comparison, the same sampling procedure must be used within each dataset and across all datasets that are being compared. Depending on the chosen 3D representation, the sampling procedure may differ (e.g., sampling of points, mesh vertices, mesh facets, or voxels). In all cases, a consistent sampling procedure should be applied together with a fixed sample size per model to ensure fair and comparable pairwise distance computation across datasets. After sampling, the metric value is calculated for each model pair. For a dataset that contains N models, this process results in an N × N matrix where each value represents the distance between two models. Because the used metrics are symmetric, the matrix is also symmetric, and the diagonal values are zero because each model has zero distance to itself.
The pairwise distance matrix captures geometric relations among all models and serves as the starting point for subsequent analyses. It can be analysed on a global level, for example, by examining the distribution of geometric distances. The same matrix can also be analysed by manifold-based embedding methods, which enable exploration of the structure of the geometric similarity space. These two analytical approaches are described in separate subsections below.
Alternatively, all models in the dataset can be analysed at the model level, allowing extraction of information about how each model differs from the others. However, this type of analysis is not the focus of this paper.
3.3.1. Distribution of geometric similarities
This analytical approach focuses on understanding the overall distribution of similarities within the datasets. The first step is to calculate the average metric value for each model within dataset separately from the pairwise distance matrix: the sum of all values in a specific row of the matrix, divided by the number of models in the dataset (reduced by 1). This allows the identification of models with the minimum and maximum average distances. Both of these models represent bounding cases: the model with the minimal value can be characterised as an average model, or as the model whose geometry is most similar to all other models, while the model with the largest average value represents a border case with the most different geometric characteristics in the dataset.
The average values for all models within dataset can be further analysed by generating a distribution curve. To enable a direct comparison between datasets of different sizes, these distributions can be normalised. Essentially, the average distance distribution provides a global overview of geometric variability within the dataset. From this analysis, it is possible to detect how diverse a dataset is, whether the majority of models share similar geometric characteristics, whether the dataset contains multiple modes that indicate different subgroups of geometries, and how synthetic or augmented datasets differ from the real industrial dataset in terms of overall variability.
3.3.2. Manifold-based embedding analysis
While the distribution analysis provides a global overview of geometric differences within datasets, it does not reveal how individual models are positioned in relation to one another. To explore this internal structure of datasets, the same pairwise distance matrices can be analysed using manifold-based embedding methods. These methods reduce the distance information into a 2D or 3D space, where models with similar geometry are positioned close to each other, and models with larger geometric differences are placed further apart. In this way, the embedding provides an intuitive visual representation of the similarity structure within the dataset.
For this purpose, several manifold-based embedding methods can be used, such as Multidimensional Scaling (MDS), Isomap, and UMAP (Reference Radvar-Esfahlan and TahanRadvar-Esfahlan & Tahan, 2013). Each of these methods projects the distance matrix in a different way and therefore emphasises different levels of geometric variation within the dataset. MDS focuses on preserving global distance relationships, Isomap additionally considers the underlying manifold structure, while UMAP highlights local neighbourhoods within the data. Using such projections, it becomes possible to detect clusters of geometrically similar models and to perform additional analysis of geometric characteristics inside each cluster.
When the selected manifold-based embedding methods were applied to the pairwise distance matrix of each dataset separately, the resulting embeddings were first characterised for each dataset individually and then compared across datasets. The comparison focused on examining the number and diversity of the identified clusters, their relative arrangement within the embedding space, and the overall structural differences between datasets.
4. Example case
The proposed 3D model dataset two-stage geometric characterization approach was preliminarily tested on an example case, which included a real-world industrial dataset and three versions of corresponding synthetic datasets to showcase the differences in the extent of geometrical variability covered by each of them. As an example of a real-world industrial dataset, a total of 1506 STL models of personalized dental abutments were used. Dental abutments are connection pieces which connect dental prosthetics (e.g. crown) with implants that are anchored in the human jawbone. The 3D geometry of these massively personalized products is characterised by complex surfaces with no obvious border between functional segments. The four examples of abutments were shown in Figure 1 b. Two examples on the left side on Figure 1 b represent the most different models within the real industrial dataset, whereas the ones on the right side are the most different within the second augmented dataset.
The synthetic dataset was generated using Rhino and Grasshopper from a specifically designed parametric CAD model by sampling the input parameters within predefined ranges. Each generated geometric variant was exported as an STL model (350 variants were generated in the example case). Two additional instances of synthetic datasets were created by rotating some of the models from the initially generated synthetic dataset. The first augmented dataset (350 models) was generated in a way that 20% of randomly selected models from the synthetic dataset were rotated around the x-axis for 180 degrees to replicate models from real-world dental abutment dataset which were designed and oriented for application on upper human yaw. Since a significant portion of models within the real dataset was stored in an orientation determined by their anatomical placement and mounting position (not only upside-down but also in other different directions), the second augmented dataset (700 models) was generated in a way that all models from the synthesized dataset were duplicated, and, after that, the duplicated half of the models was randomly rotated around each of the main axes.
As proposed in the previous section, all four datasets were aligned (translated) and normalized (within a unit sphere). After alignment and normalisation, each model was represented as a point cloud obtained by uniformly sampling 2048 points from the mesh vertices, where vertices from internal surfaces were also included for the cases where such surfaces were present in the STL tessellation. Pairwise distance matrices were then computed and analysed using two approaches: (1) the distribution of geometric similarities and (2) manifold-based embedding analysis.
4.1. Distribution of geometric similarities
The first approach focused on the descriptive statistics of pairwise distances, providing a quantitative overview of the internal geometric variability of each dataset. Figure 1 shows the distributions of the average pairwise distances (grouped into intervals of 0.015) calculated using the Hausdorff metric: blue curve corresponds to the industrial dataset, orange to the synthetic dataset, while green and red represent the two augmented versions of the synthetic dataset respectively. Hausdorff distance was selected because it highlights outlier models by capturing worst-case deviations between shapes. While metrics such as Earth Mover’s Distance can provide more representative comparisons in some settings, they are typically more computationally demanding.
In the real industrial dataset, the smallest average distance of one model compared to all others equalled 0.29 mm, while the most different model reached 0.66 mm average distance. The overall shape of the distribution curve suggested a unimodal, moderately skewed distribution with a clearly defined peak at 0.35 mm and a long tail toward higher values. The curve shape highlights the central tendency and spreads of the real industrial data, forming the reference distribution against which the synthetic datasets can be assessed.
The distribution curves of the raw synthetic dataset and the first version of augmented dataset (the one where 20 % of models were rotated for 180°) exhibited nearly identical distributions, both forming a unimodal, left-skewed shape concentrated around lower metric values and peaking at 0.29 mm, suggesting less variability within the datasets. The average distances between models ranged from 0.22 to 0.42 mm within the synthetic dataset, and 0.23 to 0.44 mm in the first augmented dataset.
a) Comparative distribution of average pairwise Hausdorff distances among datasets, b) dental abutments examples: real industrial dataset (left), augmented synthetic dataset (right)
Figure 1 Long description
Panel A: A histogram displays the distribution of average pairwise Hausdorff distances across four datasets: Real dataset, Raw Synthetic dataset, Augmented dataset v1, and Augmented dataset v2. The x-axis represents the average Hausdorff distance to all other models, ranging from 0.2 to 0.7. The y-axis shows the percentage of models, ranging from 0 to 25 percent. Each dataset is represented by a different color: blue for Real dataset, orange for Raw Synthetic dataset, green for Augmented dataset v1, and red for Augmented dataset v2. Panel B: Two images of dental abutments are shown. The left image represents a real industrial dataset, and the right image represents an augmented synthetic dataset.
Higher average distances were found within in the second augmented synthetic dataset (in which the models were duplicated and randomly rotated). Here, the distribution curve was bimodal, with peak values at the average distances of 0.42 and 0.65 mm. This dataset contained a model with a minimal average distance of 0.39 mm, while the most different model exhibited an average distance of 0.72 mm when compared to all other models within this specific dataset.
It can be observed that the first two versions of the synthetic dataset were able to capture the geometric diversity exhibited by the majority of models in the real industrial dataset, but there was a high probability that boundary cases in the real dataset were not fully covered. On the other hand, the third synthetic dataset exhibited greater internal diversity, meaning that its models could cover a wider range of geometric characteristics values of the complete industrial dataset. This analysis showed how well each synthetic dataset captures the geometric variability compared to the real industrial dataset, as reflected in the shape and spread of their corresponding distribution curves.
4.2. Manifold-based embedding analyses
While the first analytical approach provided an overall quantitative overview of the geometrical similarity within a dataset, it could not reveal the underlying structure of geometric relations between individual models. To explore the dataset structure in more detail, the second demonstrated analytical approach was the manifold-based embedding analysis using MDS, Isomap, and UMAP, all applied to the same pairwise distance matrix, which enabled examining the structure of the geometric similarity space. This space represents a high-dimensional structure defined by the pairwise geometric distances between models.
Figure 2 illustrates the geometric similarity structure for all datasets. The first row shows the results for the real industrial dataset, while the second, third, and fourth rows show the results for the three versions of the synthetic dataset (raw and augmented). For each dataset, the graphs for MDS, Isomap, and UMAP are plotted from left to right, respectively.
Using MDS analysis, the industrial dataset exhibited a uniformly distributed arrangement of models, with no clearly separated clusters, although a small subset of models was classified as outliers. Using Isomap and UMAP analyses, a similar dataset structure was observed. Structures like this indicate that the dataset cannot be separated into clusters based on geometric variability within the dataset.
The synthetic dataset had a structure similar to the real industrial dataset, but its embedding values were confined to a noticeably narrower range. This indicates, as already observed in the first analytical approach, that the raw synthetic dataset could not fully capture the geometric variation in the industrial dataset. The first augmented dataset exhibited a similar structure when characterised using MDS and Isomap. However, when UMAP was applied, the models that were rotated during the augmentation process formed a separated cluster (clearly visible in Figure 2). Such structure was expected because UMAP emphasises local neighbourhood structures, making small geometric deviations more visible in the embedding space.
In contrast to the previous datasets, the second augmented dataset exhibited a highly heterogeneous distribution. Using MDS, two separated clusters were detected, and a small portion of models was classified as outliers. Using Isomap analysis, the raw portion of the synthetic dataset is visible on the left side of the graph, while the rest of the embedding data points correspond to the duplicated models with random rotations. UMAP analysis reveals seven clusters. The first one mostly contains the synthetic dataset and models which were rotated for angle smaller than degrees, while the remaining clusters correspond to groups of models with similar values of rotation angles applied.
Comparison of MDS, Isomap, and UMAP embeddings across all datasets
Overall, this embedding analysis demonstrated that the synthetic dataset and the first augmented dataset preserved a structure similar to the real industrial dataset, although with reduced geometric diversity. The second augmented dataset introduced substantially greater variation, meaning that it could have potentially described the complete variability of the industrial dataset, but, at the same time, also contained a large number of models with large geometric differences.
4.3. Exploration of synthetic dataset performance
The presented analysis provides important insights into the geometric diversity within each dataset. However, to support the conclusions about the usability of each dataset for NN training, a PointNeXt algorithm (Reference Qian, Li, Peng, Mai, Hammoud, Elhoseiny and GhanemQian et al., 2022) was trained using different versions of synthetic data. The trained model performance was validated on the basis of a manually segmented subset of the real-world dataset. This explorative step illustrates how different levels of geometric variability in synthetic datasets, realised through different augmentation techniques, affect NN performance on real-world industrial data.
The performance of the trained segmentation NN algorithm was measured using the mean Intersection over Union (mIoU) metric (Reference Wang, Berman, Rannen-Triki, Torr, Tuia, Tuytelaars, Gool, Yu and BlaschkoWang et al., 2023). The model trained on the raw synthetic dataset achieved 96.3% mIoU when tested on a portion of the synthetic dataset. However, when applied to the real industrial dataset, performance dropped to 82.4% mIoU. From the validation set, the lowest mIoU value was 9.4%, corresponding to a model oriented differently than those in the synthetic dataset, causing poor performance. Higher performance was achieved by models trained on the augmented datasets: 88.9% mIoU for the first augmentation and 89.2% mIoU for the second augmentation on the real industrial dataset, with significant improvements on differently oriented models. This reveals that augmentation informed by 3D model dataset characterization resulted in a better performing NN model (as predicted).
5. Discussion and conclusions
The experimental case demonstrated how 3D model dataset characterization can be used to evaluate whether a synthetically generated dataset can adequately reproduce the geometric variability found in industrial data. By combining distribution analysis of pairwise distances with manifold-based embedding, the paper describes a detailed examination of both global diversity and the structure of 3D model similarity within datasets. The results show that the synthetic datasets exhibited a more compact similarity structure and lower geometric variability than the industrial data, while augmentation can increase this variability depending on the selected strategy. In the context of the research question about how geometric similarity metrics can guide the preparation of synthetic datasets without compromising NN performance, representativeness was defined as the extent to which a synthetic dataset matches the industrial distance distribution (including its upper tail) and the embedding structure, suggesting that geometric characterisation can indicate whether a synthetic dataset reflects the variability observed in real industrial models.
Although previous research highlights the importance of reproducing real geometric variability for robust NN performance, most studies did not quantify this relationship (Reference Lopez Morales, Katsimpalis, Haas and NarasimhanLopez Morales et al., 2025). Our results show that, in the example case, the base synthetic dataset captured only a limited portion of industrial variability, whereas augmentation improved performance only when targeted toward increasing diversity in ways indicated by the industrial reference. Specifically, the characterization suggested that multi-axis rotations (rather than a single-axis flip) were needed to cover orientation-driven boundary cases and better explore the underlying shape space. The example case also showed that the applied manifold-based embeddings can reveal structural patterns not detectable from global distance values alone. They exposed gaps in the similarity-space structure, boundary cases, and clusters within each dataset, aligning with earlier work demonstrating the value of embeddings for understanding high-dimensional similarity spaces (Reference Farzamnik, Ianiro, Discetti, Deng, Oberleithner, Noack and GuerreroFarzamnik et al., 2023).
To showcase the implications of geometrical characterization for ML applications, different versions of synthetic data were used to train a segmentation NN algorithm, as part of the example case. The accuracy closely mirrored the geometric characterisation: the segmentation model trained using the first version of the synthetic dataset was appropriate for synthetic models but performed poorly on the real industrial dataset, especially for rotated models. Substantial improvement was obtained when the synthetic dataset was augmented prior to training, through the combination of duplication and rotation, which provided the most appropriate geometric diversity. These results emphasise that, for robust transfer to real data, synthetic datasets should mimic dataset-level geometric variability as captured by the chosen metric(s); explicit semantic/feature coverage is not assessed here and is left for future work. Moreover, geometric characterisation can serve as a practical screening signal for selecting or augmenting synthetic subsets before allocating training resources, helping anticipate when performance drops on real data may occur.
Despite the showcased potential, several limitations should be noted. First, the proposed approach was demonstrated on a single industrial dataset, which limits the generalisability of the observed variability patterns. Second, the example case assumes that the parametric CAD model and the generated synthetic variants contain geometric features representative of the real industrial dataset; however, explicit feature/semantic coverage was not verified in this study. Third, the analysis focused primarily on within-dataset variability (internal diversity) rather than on explicitly quantifying the between-dataset gap between real and synthetic collections. Finally, the pairwise distance matrices were computed using only the Hausdorff metric, and the impact of alternative similarity metrics was not assessed.
Future work will therefore (1) evaluate the sensitivity of the characterization results to different geometric distance metrics, (2) complement the descriptive distribution comparison with formal distributional measures and statistical tests (e.g., Kolmogorov–Smirnov and Wasserstein distances), and (3) extend validation to additional industrial datasets. Further directions include integrating curvature- and feature-based descriptors to better capture task-relevant geometric properties, as well as developing automated procedures for selecting representative subsets from large synthetic datasets informed by the proposed characterization pipeline.
Acknowledgement
This research was funded by the project NPOO.C3.2.R3-I1.04.0121: Generative Design for Mass Personalization of Dental Implantoprosthetic Abutments (GENKON).
Open access
