2.1. CAD software

In today’s engineering companies, CAD is a critical software tool because it enables efficient design, analysis, and optimization. CAD allows for visual inspection of complex 3D models before prototyping, streamlining product development. CAD enhances design engineer productivity and capabilities, making design iteration and simulation easy (Reference Sharma, Sharma and ShuklaSharma et al., 2023) CAD software tools must provide a constraint solver and a solid modeling kernel. The constraint solver ensures sketch constraints are fulfilled, and the solid modeling kernel finally assembles the three-dimensional (3D) object. These two components rely on a structured sequence of parametric commands and constrained sketches. Several major CAD software vendors have developed proprietary software over decades, while open-source alternatives, like OpenCascade, provide many functions but are considered less feature complete.

2.2. CAD data

In essence, CAD data is a sequence of parametrized commands such as sketch, extrude, chamfer, or fillet. A solid modeling kernel can convert the command sequence into a geometry. In engineering design, CAD is preferred due to these parametric modeling capabilities and construction history features (Reference Vukašinovic and DuhovnikVukašinovic & Duhovnik, 2019). Constrained sketches are a crucial feature of CAD data, enabling the enforcement of geometric relations like parallelism and perpendicularity. These constraints provide high editability, allowing models to be easily modified locally while maintaining consistency by propagating changes along constraint relations (Reference SarkarSarkar, 2014). Extrusion is then used to expand the two-dimensional sketch into 3D space. The construction history captures the sequence of operations used to create the model, preserving the entire design process rather than just its outcome.

2.3. GenAI models for computer-aided design

Recent advances in deep learning have enabled learning-based methods to recreate the CAD modeling sequence history, sketch constraints, and extrusion, which can be executed in a CAD tool to generate the final 3D model (Reference Wu, Xiao and ZhengWu et al., 2021; Reference Xu, Willis, Lambourne, Cheng, Jayaraman and FurukawaXu et al., 2022, Reference Xu, Jayaraman, Lambourne, Willis and Furukawa2023). Other approaches, like Reference Jayaraman, Lambourne, Desai, Willis, Sanghi and MorrisJayaraman et al., 2023 directly generate B-REP data without relying on command sequences. Generating commands offers advantages like human interpretability and ease of editing, while direct B-REP synthesis may be the easier choice when the construction history is not required.

Current GenAI models for CAD are commonly based on the transformer architecture (Reference Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and PolosukhinVaswani et al., 2017) that outputs a sequence of tokens corresponding to the construction history’s commands and parameters, mimicking the human design process. Changes in machine learning architecture led to incremental improvements across different models, while all GenAI models remain using the same underlying training dataset. Modifications to the original transformer architecture have been made to adapt to a CAD model’s mix of data types and hierarchical structure. DeepCAD (Reference Wu, Xiao and ZhengWu et al., 2021), for example, uses separate embeddings and loss functions for CAD commands and their parameters. SkexGen (Reference Xu, Willis, Lambourne, Cheng, Jayaraman and FurukawaXu et al., 2022) uses separate transformer models to encode primitives, loops, and sketches.

5.1. Training data availability

For deep-learning models, improvement in accuracy metrics is generally correlated with larger training datasets (Reference Kaplan, McCandlish, Henighan, Brown, Chess, Child, Gray, Radford, Wu and AmodeiKaplan et al., 2020). However, most available 3D datasets for machine learning are limited in size, particularly in construction histories and constrained sketches, which are crucial for CAD modeling.

The first and still most prominent dataset resembling CAD is the ABC dataset (Reference Koch, Matveev, Jiang, Williams, Artemov, Burnaev, Alexa, Zorin and PanozzoKoch et al., 2019). This dataset is parsed from data available on PTC OnShape, a free-for-personal-use CAD platform where publicly accessible models are created by users. However, these models are neither filtered nor curated, and the dataset only includes the models in STEP file format, which lacks the constrained sketches and construction history essential for professional CAD modeling.

SketchGraphs (Reference Seff, Ovadia, Zhou and AdamsSeff et al., 2020) makes use of OnShape as well. However, in contrast to the ABC dataset, it extracts the constrained sketches but does not extrude them into 3D space and the entire construction history. The result is a dataset of 2D drawings together with dimensional and geometric constraints.

DeepCAD (Reference Wu, Xiao and ZhengWu et al., 2021) extends the ABC dataset by introducing an enhanced extraction script that captures the construction history of the models, though it is limited to only sketch and extrude operations, excluding more complex operations (see Figure 3). This results in 129,624 models available in a custom JSON and STEP format, with complete construction histories but lacking geometric constraints.

Another dataset is the Fusion360 Gallery Dataset (Willis, Pu, et al., 2021), which includes around 8,625 models sourced from the Autodesk Community Gallery and published by Autodesk researchers. These models are published in a custom JSON format, including constrained sketches and the entire construction history. Again, it is limited to sketch and extrude command types.

The characteristics of the discussed CAD datasets are compared in Table 2, highlighting substantial differences in construction history availability and primitive shape distributions.

Table 2. Comparison of CAD model datasets: size, construction history, and primitive shape distribution. Table created by authors

Figure 3. Randomly sampled CAD objects from the DeepCAD dataset by Reference Wu, Xiao and ZhengWu et al., 2021, exemplifying the simple nature of the training data. Graphic created by authors

While 3D printing, computer graphics, and similar interest groups publish CAD data collected in a dataset, this content predominantly does not represent industrial applications. As a result, a significant deficiency exists in professional-grade CAD data. This is reflected in the DeepCAD and Fusion360 Gallery datasets, containing trivial objects such as a single cylinder or cuboid. Compared to CAD models utilized in engineering firms, publicly available datasets often lack the complexity and detail characteristic of CAD models, thus limiting their applicability in professional settings. Another common issue across these datasets is the presence of geometric errors, such as self-intersecting edges, misaligned edges, and duplicate vertices, which complicate their use in training machine learning models. Finally, the CAD models within each dataset are devoid of annotations, such as captions, names, labels, or any additional information that could facilitate training for subsequent tasks such as Text-To-CAD integration.

5.2. Data representation

A key aspect of GenAI for CAD research involves developing techniques for encoding complex parametric data structures into formats suitable for machine learning.

Effective data representation is a critical factor in the success of deep learning models (Reference Goodfellow, Bengio and CourvilleGoodfellow et al., 2016). Properly structured and comprehensive representations enable the model to capture essential features, improving learning efficiency, generalization, and overall performance in each task.

A significant challenge in CAD modeling stems from its sequential and parametric nature. The order of operations is critical, as each operation must be executed in a specific sequence (Reference Wu, Xiao and ZhengWu et al., 2021). Furthermore, for each operation, the generative AI model must identify the appropriate parameterization to ensure accurate representation and functionality of the design.

The parametrizations of each sequence step are, in turn, challenging as the parameters are a mix of discrete data (for example, clockwise sign), continuous data (for example, extrusion distance), and relational data (for example, parallel constraint between two lines). Relational constraints can exist between various elements, such as edges (e.g., collinear constraints) or points (e.g., coincidence constraints). The added complexity arises from handling constraints between edges and points on those edges, necessitating advanced strategies for effectively representing and maintaining these relationships within the model.

Current research (Reference Wu, Xiao and ZhengWu et al., 2021; Reference Xu, Jayaraman, Lambourne, Willis and FurukawaXu et al., 2023) has simplified the representation of CAD data by omitting constraints. This method represents a model’s construction history by encoding CAD commands and their parameters into sequential vectors. Although this method is computationally efficient, it overlooks constraints—an essential feature in CAD.

5.3. Proprietary file formats

Commercial CAD vendors protect their data files through proprietary binary formats that are accessible only via licensed software, keeping innovation in the hands of the commercial vendors (Reference Stroud and NagyStroud & Nagy, 2011). The licenses are often costly, limiting accessibility for academic researchers and startups. Additionally, incompatible file formats across different vendors create artificial barriers, hindering data aggregation from diverse sources and generalizing research findings. A neutral format, in contrast, would provide independence from vendor-specific formats, promoting broader interoperability and compatibility. While the STEP and IGES formats aim to improve interoperability, they come at the expense of losing constrained sketches and construction history information.

Existing research has introduced custom JSON formats to represent CAD model information, but their structure remains closely tailored to the specific scope of each project (Reference Willis, Pu, Luo, Chu, Du, Lambourne, Solar-Lezama and MatusikWillis et al., 2021; Reference Wu, Xiao and ZhengWu et al., 2021). File format of open-source alternatives like FreeCAD is as well closely coupled with the main software and is not designed for interoperability. Defining a comprehensive CAD format is challenging, as it must represent geometric shapes and encode details such as tolerances, materials, and annotations.

5.4. CAD operations

Current generative AI approaches for CAD modelling primarily rely on Sketch and Extrude operations, which represents a significant limitation for practical applications. While these operations form the foundation of CAD modelling, professional designers routinely employ a broader set of operations including Revolution, Sweep, Loft, Blend, Fillet, and Chamfer to create complex geometries (Reference Heidari and IosifidisHeidari & Iosifidis, 2024). Operations like Fillets and Chamfers can only be applied to B-REP edges—elements that only emerge after converting the Sketch and Extrude sequence into a B-REP model (Reference Jayaraman, Lambourne, Desai, Willis, Sanghi and MorrisJayaraman et al., 2023). This technical constraint forces designers to manually add these features as post-processing steps, diminishing the efficiency gains promised by generative AI. While the Sketch and Extrude paradigm provides a useful starting point for research, recent work on highlights the necessity of incorporating a wider range of operations to generate realistic engineering components (Reference Li, Guo, Zhang and YanLi et al., 2023; Reference Zhang, Guan, Jiang, Ning, Wang and TanZhang et al., 2023).

6.1. GenAI architecture for CAD data

A critical challenge in developing generative AI models for computer-aided design is effectively encoding the mix of categorical, continuous, and relational data types. As it is difficult for a single model to comprehend a mix of data types (Reference Borisov, Leemann, Seßler, Haug, Pawelczyk and KasneciBorisov et al., 2024), we propose a mix of expert models (see Figure 4) to capture each of the essential features of the CAD object because it may be challenging for just a single model to handle different types of data. However, the implementation of expert model ensembles will likely still require domain-specific fine-tuning or retraining to accommodate the diverse use cases described in Section 4. This comes with the practical constraint of requiring additional computational resources.

Our method assumes that CAD objects are constructed through an alternating sequence of two fundamental operations: sketch and extrusion. Each of these operations is handled by dedicated generators, which take a conditioning vector as input to control the generation process. The conditioning vector (Reference Mirza and OsinderoMirza & Osindero, 2014) can vary depending on the specific use case and could be derived from various sources such as embeddings of feature specifications, images, or any other relevant information. This flexibility allows our method to adapt to different requirements and generate CAD objects based on the provided conditioning vector input. The sketch generator consists of two submodules: a primitive generator and a constraint generator (Reference Para, Bhat, Guerrero, Kelly, Mitra, Guibas and WonkaPara et al., 2021).

Figure 4. Schematic of our proposed GenAI architecture for CAD with separate sketch and extrude generators for autoregressive generative of CAD command sequences. Graphic created by authors

The primitive generator is a decoder-only transformer (Reference Radford, Wu, Child, Luan, Amodei and SutskeverRadford et al., 2019) that produces lines, arcs, and circles through an autoregressive process to accommodate variable-length sketches. Consistent with previous work (Reference Wu, Xiao and ZhengWu et al., 2021; Reference Xu, Willis, Lambourne, Cheng, Jayaraman and FurukawaXu et al., 2022, Reference Xu, Jayaraman, Lambourne, Willis and Furukawa2023), we employ the technique of quantizing continuous parameters into discrete values. However, in our approach, we utilize a higher level of numerical precision to achieve more accurate results.

Constraints between edges of a sketch are well represented as a constraint graph (CG), where primitives serve as nodes, and constraints in the sketch relate to annotated edges in the CG (Reference DingDing, 2014; Reference Seff, Ovadia, Zhou and AdamsSeff et al., 2020). Binary constraints such as parallel or perpendicular can be directly mapped from and to the CG. For unary constraints such as vertical and horizontal, we introduce virtual nodes representing coordinate axes, allowing these constraints to be modelled as binary constraints in the CG.

Constraints such as coincidence and concentric don’t operate on primitives but on the endpoints of primitives, making a separate CG necessary to generate coincidence and concentric constraints. Other constraint types exist, but the mentioned constraints already cover most cases as seen in Figure 5 While earlier methods employed pointer networks to model CGs (Reference Seff, Zhou, Richardson and AdamsSeff et al., 2022), our constraint generator is a Graph Neural Network (GNN) to fully leverage the inherent graph structure. For training, we assemble CGs from existing CAD objects. At inference, our GNN acts as a link predictor using primitive nodes for input and outputting a CG. Link prediction using machine learning has been demonstrated in Reference Zhu, Zhang, Xhonneux and TangZhu et al., 2022.

The extrusion generator is also a decoder-only transformer (Reference Radford, Wu, Child, Luan, Amodei and SutskeverRadford et al., 2019) and determines appropriate extrusion parameters, for example extrusion distance or taper angle based on sketch geometry, construction history, and conditioning vectors. Again, continuous parameters quantized.

Instead of using end-of-sequence tokens within each generative module, our architecture employs a dedicated stopping classifier. This modular approach separates the generation logic from termination decisions, allowing each component to focus on its specific task while the classifier independently determines when the model has produced a complete CAD object based on the construction history and conditioning vector.

Combining sketches and extrude parameters creates a construction sequence that can be converted into a 3D object by a solid modelling kernel. This approach enables the generation of complex, manufacturable CAD models. To manage model complexity, we propose to pre-train each module separately and then fine-tune the entire ensemble on a smaller data set.

Figure 5. Distribution of constraints in the Fusion360 gallery dataset by Reference Willis, Pu, Luo, Chu, Du, Lambourne, Solar-Lezama and MatusikWillis et al., 2021. With the mentioned constrains about 93.6% of all constraints are covered by our proposed GenAI model

6.2. Accumulation of a large dataset

Engineering and manufacturing companies create detailed CAD models as technical specifications for production but are reluctant to make these models publicly available. This presents research opportunities to address the lack of accessible industrial CAD data while considering company requirements. Potential solutions include data anonymization through feature removal and geometric transformations, and the generation of synthetic datasets. However, machine-learning models trained on synthetic data typically achieve lower accuracy when validated against real data (Reference Rankin, Black, Bond, Wallace, Mulvenna and EpeldeRankin et al., 2020).

By pursuing these research directions, academia can help bridge the gap between industry and research in professional-grade modelling while respecting the proprietary nature of industrial CAD data. Since industry cooperation is crucial for data access, this could involve exploring collaborative opportunities and developing data-sharing mechanisms that benefit both research institutions and companies.

6.3. Expanding CAD command palette

Current GenAI methods for CAD primarily focus on generating solid-body components with the two basic operations sketch and linear extrusion, limiting their applicability to real-world design tasks. Future research should prioritize expanding these methods to support a broader range of complex modeling operations commonly used in industry, such as fillet, chamfer, loft, sweep, and draft (Reference Heidari and IosifidisHeidari & Iosifidis, 2024). Sheet metal design, which heavily relies on bending operations, is also notably absent from existing work despite its prevalence in automotive manufacturing (Reference TrzepiecińskiTrzepieciński, 2020).

For sketch primitives current state-of-the-art models such as Reference Wu, Xiao and ZhengWu et al., 2021; Reference Xu, Willis, Lambourne, Cheng, Jayaraman and FurukawaXu et al., 2022, Reference Xu, Jayaraman, Lambourne, Willis and Furukawa2023 only support line, arc and circle primitives in both their datasets and GenAI models (Reference SarkarSarkar, 2014). The accumulation of a large dataset as proposed in chapter 6.2. could include new commands besides sketch and extrude, which would be essential for developing more versatile and industry-relevant GenAI CAD systems.

6.4. Vendor-neutral file format

A vendor-neutral CAD file format would foster innovation by eliminating barriers created by proprietary formats and incompatibilities between CAD systems (Reference Lee, Baek and OhLee et al., 2019). A neutral format would enable broader interoperability, making aggregating data from diverse sources easier. Achieving this could be accomplished either through a collaborative standardization process or by the widespread adoption of a single open-source format. Developing a standardized format upon existing JSON formats or YAML could provide significant benefits since both are flexible, human readable and version control compatible make it a promising foundation for CAD data exchange. Its widespread support across programming languages could facilitate broader adoption and collaboration in the CAD community.