1. Introduction
Virtual reality (VR) enables immersion in a computer-generated, three-dimensional environment (Reference Weidlich, Cser, Polzin, Cristiano and ZicknerWeidlich et al., 2007). VR’s distinguishing feature is its high level of immersion, achieved through visual representation, spatial sound, and haptic feedback (Reference FuchsFuchs, 2017).
The integration of computer aided design (CAD) into VR enables engineers, architects and designers to create and modify 3D models spatially at a realistic scale and with immersive interaction. This gives designers the ability to enhance existing desktop CAD applications with aspects of spatial perception (Reference Bourdot, Convard, Picon, Ammi, Touraine and VézienBourdot et al., 2010; Reference Feeman, Wright and SalmonFeeman et al., 2018). As illustrated in Figure 1, VR facilitates solid modeling, requirement changes, data transformation, and component integration, enhancing technical knowledge transfer. Studies highlight VR’s ability to boost creativity, immersive manipulation, and proportion accuracy in professional design (Reference Stark, Israel and WohlerStark et al., 2010; Reference Feeman, Wright and SalmonFeeman et al., 2018), while expanding engineering design capabilities and providing deeper insight into complex processes (Reference Lorusso, Rossoni and ColomboLorusso et al., 2020). The combination of CAD, VR, and optimization tools can transform product development into an efficient environment that increases efficiency and better supports product development. Modern VR CAD tools integrate established software like Siemens NX and eDrawings through CAD plug-ins (Reference Praj, Prajová, Košt”ál and CambalPraj et al., 2023). However, combining VR and CAD faces challenges.
Both technologies demand substantial computing resources. VR’s polygonal surface data lacks the topological and precise geometric details (e.g., B-rep) of CAD formats like STEP, leading to conversion artifacts, gaps, and accuracy loss (Reference Montagnat, Delingette and AyacheMontagnat et al., 2001; Reference Hurtado, Montenegro, Gattass, Carvalho and RaposoShah et al., 2001; Reference Shah, Anderson, Kim and JoshiHurtado et al., 2022; Reference Raposo, Corseuil, Wagner, F. and GattassRaposo et al., 2006). Repair algorithms exist but may alter geometry, reducing model fidelity. Additionally, CAD precision achieved with desktop interfaces is hard to replicate with VR controllers, necessitating customized, intuitive user interface (UI) design for VR environments. Addressing usability, data accuracy, UI, and accessibility is crucial for broader professional adoption (Reference Mine, Yoganandan and CoffeyMine et al., 2015). A new framework for CAD in VR is introduced in this paper to tackle the aforementioned problems. The CAD kernel Open CASCADE (OPEN CASCADE S.A.S, 2024) is integrated in the game engine Unreal Engine (Epic Games, Inc., 2024) to create a standalone application for VR devices that is independent of external hardware or software.
Figure 1. Objectives of CAD solid modeling in virtual reality
Note: Logos of the humans with VR spectacles and 3D Axis has been taken from http://www.freepik.com/
Primary CAD functions are implemented and can be directly performed in the VR environment.
To design CAd components in VR and as a proof of concept, a new user interface was developed that is atypical for desktop applications.
The contributions of this work can be summarized as follows:
-
A novel architecture for standalone CAD application in VR is introduced and implemented.
-
Basic CAD functions such as creating, editing and topologically altering primitives are demonstrated as well as importing and exporting STEP files.
-
A new UI for handling CAD operations in VR is presented.
The concept of immersive 3D modeling in VR, shown in Figure 1, aims to benefit designers, engineers, and customers by enabling real-time design changes and functional testing in a virtual environment. This approach enhances customer feedback, iterative refinement, and design understanding without physical prototypes. In line with the goal of the paper, it integrates VR into CAD workflows, creating an efficient design solution that can be utilized by designers as well as end users.
To establish the research focus of this paper, an in-depth analysis of the current scientific and industrial work is provided in Section 2. The problem definition and derived requirements are presented in Section 3. A detailed explanation of our methodology is provided in Section 4. The application of our methodology and its results are discussed in Section 5. The paper is concluded in Section 6.
2. State of the Art
In this section, representative approaches, methods, and programs are described, and applications relevant to VR are listed. State-of-the-art applications dealing with technical CAD components and their interaction are presented in Table 1. The examined commercial applications are eDrawings (Dassault Systèmes SolidWorks Corporation, 2024), Siemens NX-VR (Siemens Digital Industries Software, 2024), Halocline (Halocline GmbH & Co. KG, 2024), VR Sketch (Baroque Software, 2024), ELITECAD VR (XEOMETRIC GmbH, 2024), Mindesk VR (Mindesk Inc., 2024), Sky-Real (Skyreal, 2024), Gravity Sketch (Gravity Sketch, 2024), and The Wild (The Wild by Autodesk, 2024). Programs that could be tested are listed in the upper half, the others are listed in the lower half with information retrieved from their respective product portfolios and partly from (Tran et al., 2021). Table 1 lists features related to the direct handling of technical 3D and B-Rep models as evaluation characteristics. Key evaluation criteria include the ability to edit, manipulate, and export topological data. Importing B-Rep based STEP files into VR scenes is common and state of the art. Most programs provide immersive, detailed representations of components. Since the full scope of some products is unclear, the label “N/A” indicates uncertain areas. More than half of the programs support the export of VR-modified components in STEP format, but none of the programs are usable on standalone VR devices. Product examples and their key features are highlighted in the following section.
Table 1. Overview of 3D modeling VR software
* Information not available.
2.1. VR CAD Applications
Siemens NX VR and eDrawings VR enable engineers and designers to visualize, review, and present CAD models in an immersive, life-size virtual environment (Siemens Digital Industries Software, 2024; Dassault Systèmes SolidWorks Corporation, 2024). Models are loaded from desktop applications and accessed via VR headsets, allowing grabbing, cropping, and scaling but no direct modification within the VR scene. Siemens NX allows note creation within VR, which can be saved and restored with the scene. While it supports simulating production processes and analyzing human-machine relationships, VR application is limited to visualization; simulations are run on the desktop applications and then transferred to the headset.
Programs like Halocline and The Wild enhance CAD and design processes with unique features tailored to various stages of design and development (Halocline GmbH & Co. KG, 2024; The Wild by Autodesk, 2024). Halocline focuses on industrial applications, enabling virtual factory planning, workflow simulations, and production layout optimization (Halocline GmbH & Co. KG, 2024). It supports designing factory layouts in VR, placing machines, production lines, and workstations, and facilitates CAD data import/export in formats like STEP or IGES. The Wild specializes in architecture, construction, and design, offering VR/AR collaboration with support for CAD formats usable in Revit and SketchUp (The Wild by Autodesk, 2024). Both tools enable creating complex structures in VR but lack sketch modes often used in CAD and they require desktop PC connection for operation. VR Sketch, a plugin for SketchUp, allows immersive viewing and editing of models while retaining typical CAD features via concurrently running desktop software (Baroque Software, 2024). Similar to CAD programs such as SolidWorks, SiemensNX or Autodesk, any base area can be created in sketch mode and then designed into 3D models based on features. Data format import and export is primarily designed for SketchUp formats, but custom plug-ins provide conversion to common CAD exchange formats for import and export. While with VR Sketch, technical 3D CAD modeling using classic CAD programs is possible in the VR scene, the SketchUp desktop application must run simultaneously. Also, the user interface is based on the SketchUp desktop application, making it challenging to add numbers or written descriptions to designed solid models.
2.2. Studies on VR CAD Application
Challenges in implementing CAD solid modeling design functionality in VR have been identified and addressed in several publications. Performance issues with complex models, inaccuracy of model geometry created in VR, and loss of information due to data transformations are some of the existing challenges (Reference Raposo, Corseuil, Wagner, F. and GattassRaposo et al., 2006).
Empirical studies show that CAD solid modeling in virtual reality is technically feasible and offers potential for increased creativity and improved spatial understanding (Reference Feeman, Wright and SalmonFeeman et al., 2018). These benefits arise from the immersive 3D representation, which allows intuitive perception and direct manipulation of model geometry. Prototypical approaches such as the integration of the Fusion 360 server with the Stingray client (Reference Feeman, Wright and SalmonFeeman et al., 2018) and the connection of the CAD software cRea to VR systems (Reference Martin, Masfrand, Okuya and BourdotMartin et al., 2017) demonstrate the technical feasibility, and allow direct interaction with CAD functions via VR controllers. Empirical studies with industrial designers and design students confirm the potential of VR as a complement to desktop CAD systems (Reference Stark, Israel and WohlerStark et al., 2010). Tran et al. (2024) describes the usage of a CAD Kernel in combination with a VR engine. This approach is based on the idea that instead of using an additional desktop application, the VR engine can communicate with the kernel in the backend via a suitable interface. However, the necessary interface to be designed here can lead to restrictions with regard to the interface configurability.
Immersive model editing allows for creative freedom, but there are challenges in terms of accuracy and usability of the user interface. Desktop applications remain superior in terms of accuracy, as established input methods such as mouse and keyboard offer greater precision. Despite these limitations, the majority of participants supported the use of VR as a supporting tool (Reference Stark, Israel and WohlerStark et al., 2010). Overall, the studies show that VR-based CAD modeling offers significant advantages for designing and editing 3D models. Challenges lie in the user interface and the precision of the input methods. The direct connection of CAD kernels to VR systems (Reference Martin, Masfrand, Okuya and BourdotMartin et al., 2017) and the use of hybrid modeling approaches (Reference Stark, Israel and WohlerStark et al., 2010) form a promising basis for the further development of immersive CAD modeling. The studies show that CAD modeling in VR offers technical and ergonomic potential. Immersive 3D representations promote creativity and spatial understanding, while challenges exist in terms of usability and accuracy.
3. Requirements for CAD in VR
In the following, we discuss the dependency of VR applications on additional desktop applications, and the need to implement freely configurable user interfaces for technical 3D design features in 3D space. The basic requirements for the system structure to be developed are derived from the respective findings.
CAD data processing capabilities
The transformation from a polygon-based model to a solid model leads to a loss of information and accuracy (Reference Raposo, Corseuil, Wagner, F. and GattassRaposo et al., 2006; Reference Botsch, Pauly, Kobbelt, Alliez, Lévy, Bischoff and RösslBotsch et al., 2007). Therefore, it is important to keep all topological model information within the data format. In order to manipulate a solid model in VR, the corresponding design functions must be available as well as the data processing capabilities (Reference Lorusso, Rossoni and ColomboLorusso et al., 2020).
Independency of CAD solid modeling from external hardware and software
With the development of standalone VR headsets such as the Meta Quest 3, it is now possible for the device itself to perform the data processing through an integrated processor and graphics card, completely bypassing the challenge of data transfer via cable or Wi-Fi, and thereby reducing the hardware required at the point of use. Therefore, placing the data processing software on the same device for solid modeling in VR is another requirement in this case.
Configurability of user interface
Reference Lorusso, Rossoni and ColomboLorusso et al. (2020) describes that the execution of design functions in space with a user interface design similar to desktop applications leads to lower acceptance of this technology.
Reference Mine, Yoganandan and CoffeyMine et al. (2015) highlights the ergonomic problems of user interaction, the inability to perform precise manipulations, the lack of appropriate system controls, and user menus in the application of solid modeling CAD in VR.
In order to be able to design in 3D virtual space in a comfortable, efficient, and intuitive way, the design of the user interface for creating solid models for engineering purposes has to be fundamentally rethought.
In summary, a freely configurable user interface is required.
4. System Architecture for Standalone VR CAD Application
The overall structure of the developed VR CAD application and the data processing integration into the VR Engine are shown in Figure 2. On the left side, data processing and preparation are done separately from the VR environment, with interaction examples given in Section 2.1. To design the VR-CAD application as a standalone solution, the integration of topological data processing must be part of the VR environment, as shown on the right side of Figure 2. If the VR scene is created by the engine and accessible via an interface, components can be created and edited within the VR scene by implementing CAD kernel data processing.
Figure 2. Visualization of current (left) and proposed (right) conceptual system structure
Current VR CAD solid modeling programs rely on topological data processing through a desktop application. Although the user can interact with visualizations of engineering components in virtual space, the VR headset and controller serve only as a replacement for desktop devices such as a mouse and keyboard. The interaction design in virtual reality is largely based on the interaction methods of the desktop application. This can limit the efficiency and intuitiveness of design methods in 3D space, as the user interfaces and interaction elements of conventional desktop applications are primarily optimized for use in 2D environments. Transferring these approaches directly into 3D space makes the development of a natural and intuitive operation of CAD functions in virtual reality more difficult. The integration of independent topology processing in VR devices through the use of a CAD kernel (see Figure 2) enables the development of specific design features independent of desktop CAD applications. This opens up new possibilities to design and interact with CAD functions directly in virtual space.
The focus should be not only on the procedural aspects, but also on the design visualization of the features and the design of the input devices themselves. Figure 3 shows two basic implementation approaches for CAD solid modeling that work in VR.
In principle, both models can be designed to be desktop independent and thus run on a standalone device, but the left-hand illustration is mainly used in the form of a parallel desktop application as described in Section 2.1. The difference between the two models is that in the left approach, data processing is done in parallel with the VR engine. This requires the development of an additional communication interface that passes the design requests from the VR input devices (e.g. controllers) to the CAD data processing. Although this approach allows for a flexible design of the user interaction, the additional interface adds complexity to the information processing. This can increase the development effort as well as the computational requirements for synchronously processing user actions and CAD functionalities. In the approach on the right, the CAD Kernel OCCT is integrated into the VR engine. The integrated controls and process-oriented infrastructure of the VR engine are used to integrate the functionalities of the CAD kernel. There is no need to develop an additional data communication interface. The necessary data processing operations can be initiated directly from the VR environment. The user interface is designed using VR-suitable methods already provided by the VR environment. Therefore, the development of the VR CAD program described in this paper is based on this structural approach.
Figure 3. From cross-program to internal communication and data transfer
For the CAD kernel, we chose the open source kernel OpenCASCADE (OPEN CASCADE S.A.S, 2024). With an open source CAD kernel, we have access to the necessary data processing and design features to perform the necessary geometric modeling tasks. These tasks include geometric data modeling, geometric and topological algorithms such as Boolean operations, and providing data formatting for compatibility with Unreal engine (Epic Games, Inc., 2024) as well as other programs that may use the created 3D models. Unreal Engine is the framework of the program and is used to perform major tasks such as creating the VR scene, rendering mesh data, and providing the user interface. It is a powerful game engine that has been widely used for game development, and graphics applications, and provides good support for VR application development. In addition, Unreal Engine provides native support for CAD formats in both editor and runtime modes through its Datasmith plugin (Epic Games, Inc., 2024). By using the Open CASCADE kernel without strict interface boundaries, it is possible to design the user interaction features freely. The requirement to integrate the kernel on the device and make it interact with the VR engine was solved by embedding the kernel into the application developed in Unreal Engine. In this way, the engine can freely access the functions of the kernel without having to transfer additional information or metadata via an additional interface.
4.1. Embedding CAD Kernel in Game Engine
To incorporate the OpenCASCADE Technology kernel (OCCT) in Unreal Engine, custom C++ classes are created which call functions from the geometry library by importing them. This configuration stands in contrast to using an API where a separate CAD program is run, and operations take place over an intermediate interface. To package the program for standalone use, the CAD kernel has to be compiled for use in Android, and subsequently, integrated into the Unreal Engine application. In the following, some key OCCT packages are presented that are used to enable CAD operations in our VR CAD program.
The TopoDS package provides methods to manage the data structure for the topological and geometric data handled in OCCT. TopoDS_Shape is the top class containing the geometric shape, topological relationship, and local position and rotation of a created 3D object. TopoDS_Shape is used as the main object for creating and manipulating CAD data in our work as well. The BRepPrimAPI package contains methods for creating primitive 3D objects such as boxes, cones, cylinders, etc. For instance, a box can be created by calling BRepPrimAPI_MakeBox() with the length, width, height (x, y, z lengths) as input arguments. The function creates a box with a corner at the origin (0, 0, 0) and corresponding lengths. The created shape is again a TopoDS_Shape object.
For Boolean operations, the BRepAlgoAPI package is used which includes methods for union, intersection, and subtraction. To create the union of two shapes, BRepAlgoAPI_Fuse() can be called with the two shapes as input arguments. The BRepMesh package contains methods for generating meshes from the geometric shapes. We use the BRepMesh_IncrementalMesh() class to build meshes for our objects, which is essential for displaying the object in VR. The created B-Rep shapes can be saved in STEP format through the STEPControl package. The STEPControl_Reader reads STEP files and translates them to OCCT objects. Conversely, OCCT objects can be saved in STEP format with the STEPControl_Writer class. The StlAPI package provides support for the STL data format. Reading STL files is enabled through the StlAPI_Reader class and meshes can be written as an STL file through the StlAPI_Writer class.
4.2. User Interface
The functions related to the manipulation of a single solid model component are shown in Figure 4, including the layout of the controller inputs of the Meta Quest 3.
Figure 4. Component manipulation with corresponding controller layout
The development and design of a suitable and user-acceptable novel UI requires free configurability of the UI in the VR CAD program presented here. The UI presented in this work is therefore a proof of concept, which on the one hand describes an alternative handling of a CAD application and on the other hand proves the free configurability of the UI. The further development of this UI will be addressed in the final discussion of the program.
In order to use the same controller inputs for different functionalities, operation modes are implemented, consisting of size, translation, rotation, scale and unselected. In addition, default functions are defined that can be accessed in any of the modes, and which are also the only functions available in the unselected mode. The user can cycle through the modes by pressing the left trigger. The positive X-direction of the Unreal Engine’s left-handed coordinate system is taken as our default front view, and the functions are mapped so that the operations performed can be intuitively navigated. In our setting, the left hand acts as the main control hand, with the left thumbstick controlling actions on the XY plane and the X and Y buttons controlling actions related to the Z axis. The A button on the right hand controller is used to access a widget menu, shown in Figure 4, which provides various CAD operations. The right thumbstick is used to navigate the widget, and the right trigger button is used to select the action. The right grab button can be used to save the model as a STEP file.
5. Demonstration of Design Process in VR
An example of creating and manipulating a box in VR is presented in Figure 4. The box is created at the world origin and can be manipulated according to the selected operation modes. In size mode, the geometric parameters of length, width and height are edited and saved. By translating and rotating the box, its position and orientation can be modified for viewing purposes as well as geometric operations such as Boolean operation with other objects. Scaling does not influence the geometric parameters and merely changes the representation in the VR environment.
Figure 5. Example manipulation of an imported STEP-file with subsequent STEP-file export
5.1. Data Import, Edit and Export
A key feature for working on technical designs is the import and export feature. Not all components in a design process are created by the user, and standard parts or purchased parts need to be integrated or considered. In Figure 5, an aluminum profile, which is a purchasable part, is loaded into the VR CAD, cut to a specified length and exported as a STEP file. Cutting the aluminum profile is in this case executed as an intersection between the original part and a created box. The exported file can be viewed in standard CAD programs (here CAD Assistant) as it is in STEP format. This is essential for the design process as with the B-Rep data in the STEP file other CAD programs can continue to work on the component as any other part created in a native CAD environment.
5.2. Summary of the developed Standalone VR-CAD Prototype
Overall, we have successfully integrated an open source CAD kernel into a VR engine, allowing STEP files to be loaded and edited directly within the VR environment. This approach eliminates the need for additional software or external computing devices such as desktop PCs, servers or processors. By leveraging Unreal’s native features, CAD kernel operations are efficient and smooth. File export is fast and reliable, allowing STEP files to be seamlessly opened and edited in conventional CAD applications such as SolidWorks. This ensures full compatibility with established CAD workflows and facilitates seamless data exchange.
While the core CAD functionality is operational, the ability to set fixed, precisely defined dimensions is not yet fully implemented. This feature is a key focus for future development as it will enhance the accuracy and usability of the VR CAD environment. With the standalone VR CAD application, users are no longer tied to a desktop. The ability to walk around and inspect parts from different angles provides real-time design feedback that can be incorporated directly into the model. Initial tests with the Meta Quest 3 have confirmed that basic CAD operations can be performed with its processing power.
Several challenges remain for future development and practical use. Since only a CAD kernel is embedded, and not a full CAD program, the complete framework and interface for CAD operations must be developed within the game engine. In addition, the lack of standard keyboard and mouse interactions requires a rethinking of user interface and user experience design. Key areas for further research include:
i. Extend CAD functionality from part editing to multi-component assembly capabilities.
ii. Design the user interface and experience for intuitive navigation to improve the design workflow.
6. Conclusion and Outlook
6.1. Conclusion
In line with the advancement of VR technology, we demonstrate a standalone VR application for immersive CAD. Based on an analysis of existing VR CAD applications and studies, requirements for a standalone VR application are derived. The requirements are the ability to process CAD data, the independence from external or additional software sources, and the freely configurable design of user interfaces. By directly embedding the open source CAD kernel OpenCASCADE into the VR Engine Unreal, additional software and cross-program interfaces are avoided and a free configurable user interface of CAD features was made possible.
CAD functions could be tested without the use of a desktop PC or an internet connection, and it was possible to export new or modified 3D models in STEP format from the VR application. This approach and its future developments will be valuable not only for industry, but also for scientific educational institutions, where participants can learn the benefits of 3D modeling directly in virtual reality.
6.2. Outlook
It is now possible to gradually integrate other essential CAD functions. Planned improvements include volume creation through translation and rotation, sketch editing, and features such as undo and redo as already recognized in Reference Feeman, Wright and SalmonFeeman et al. (2018) as necessary for CAD in VR. The user interface design will be further explored with a focus on exploiting the full virtual space and overcoming the limitations of the controllers in an intuitive and comfortable way.
In terms of knowledge transfer, natural perceptual habits in 3D space can be exploited due to the high degree of immersion, and a possible facilitated access to technical product development and process methods can be examined and evaluated.
Similar to the development of modern desktop CAD applications, functional enhancements must now be added incrementally and in an appropriate manner. To reach the level of functionality of modern CAD applications, further research and development will require a solid program foundation on which to build, as well as a consistent, extensible, and intuitive user interface. These results need to be evaluated in future research in the form of user studies in comparison with desktop CAD applications.
Acknowledgements
The authors would like to thank the Unreal Engine and Open CASCADE communities for the essential tools, resources, and support that have been essential to our work on immersive CAD modeling.
Open access
