1. Introduction
In an ideal learning scenario, lessons would be individually tailored to the needs of the learners. Learning content would be worked through at the learner’s own pace and supplemented by personalised feedback. However, engineering degree courses are dominated by teacher-centered lectures, which leave less room for interaction and individual attention to different levels of knowledge (Reference Albers, Burkardt and OhmerAlbers et al., 2004). The heterogeneity of the students, due to different previous experiences and educational backgrounds, exacerbates this problem (Reference BhathalBhathal, 2016; Reference Eckert, Seifried, Spinath and RheinländerEckert et al., 2015).
In order to counteract the heterogeneity of students in teacher-centered teaching formats, the learning levels could be individually adapted (Reference Kossack, Kattwinkel and BenderKossack et al., 2022). One-to-one tutoring offers an ideal scenario for this: individual needs and skills are specifically promoted through one-to-one support, which enables effective and customised learning (Reference BloomBloom, 1984; Reference Graesser, Person and MaglianoGraesser et al., 1995). Despite its advantages, such as the targeted treatment of comprehension difficulties (Reference WittwerWittwer, 2008), this format can hardly be realised in higher education due to limited resources.
Intelligent tutoring systems (ITS) offer a solution, enabling one-to-one support through immediate assistance and feedback without human intervention (Reference MohammadMohammad Bagheri, 2015). They monitor learning progress, recognise strengths and weaknesses and adapt learning materials individually. In this way, they offer effective, personalised learning and achieve comparable results to human tutors in their subject area (El-Sheikh & Sticklen, Reference El-Sheikh, Sticklen and Cerri2002; Gembarski & Hoppe, Reference Gembarski, Hoppe and Andersen2022; Kulik & Fletcher, Reference Kulik and Fletcher2016; VanLehn, Reference VanLehn2011).
Many ITS are already being used successfully in various domains. A central aspect is the automatic analysis of user input, for example by comparing individual solutions with sample solutions in mathematics or the functional testing of solutions in computer science, regardless of the solution path (Reference Kulik and FletcherKulik & Fletcher, 2016).
Design education places special demands on an ITS, as problems can often be solved by numerous variants, which requires a comparison with a solution space (Jaakma & Kiviluoma, Reference Jaakma and Kiviluoma2019; Otto & Mandorli, Reference Otto and Mandorli2018; Plappert et al., Reference Plappert, Hoppe, Gembarski and Lachmayer2020). In addition, solutions must be evaluated in the context of different application areas, e.g. manufacturing, functionality and assembly, which requires a context-sensitive system design. A system that adapts dynamically to context by processing environmental data (e.g., task, user behaviour, shaft section) to enhance user experience, automate processes, and enable real-time adjustments (Reference Vieira, Tedesco and SalgadoVieira et al., 2011). The aim of this article is to develop an ITS that fulfils these requirements.
2. Theoretical background
ITS have been used for decades as adaptive learning systems that mimic human tutors (Reference FletcherFletcher, 2006). Their application covers diverse areas such as grammar (Reference SchwindSchwind, 1990), mathematics (Reference Anderson, Boyle and ReiserAnderson et al., 1985; Reference Spitzer, Moeller and MusslickSpitzer et al., 2023), programming (Anderson et al., Reference Anderson, Boyle and Reiser1985; Schez-Sobrino et al., Reference Vallejo, Glez-Morcillo and Redondo2020) and medicine (Reference Almurshidi and Abu-NaserAlmurshidi & Abu-Naser, 2017). When using these systems, there are four basic characteristics that are crucial for any adaptive learning system (Reference MohammadMohammad Bagheri, 2015):
1) A paedagogical framework justifies the choice of activities and makes decisions about how to proceed. It is based, for example, on one of the common learning theories such as behaviourism, cognitivism and constructivism.
2) The system provides immediate, individualised feedback depending on the level of learning, for example by correcting errors.
3) The learner is assessed by analysing the individual results.
4) The system then enables the task to be revised and thus provides the opportunity to improve the solution.
Most ITS utilise the basic architecture from Matthews shown in Figure 1. According to this, an ITS consists of a student, a tutor and an expert model as well as a user interface.
Figure 1. Architecture of a ITS according to (Reference MatthewsMatthews, 2013)
The expert model contains the specialised knowledge of the content to be taught and serves as a benchmark for the performance and knowledge level of the learners (Reference MatthewsMatthews, 2013). It includes theoretical material, tasks, explanatory units as well as rules and principles of a domain (Reference AnohinaAnohina, 2014). Declarative knowledge represents facts about the subject area, while procedural knowledge comprises rules and arguments for solving problems.
The learner model captures the learner’s level of knowledge, identifies misconceptions and supports their correction (Reference MatthewsMatthews, 2013). It uses data from problem-solving behaviour, historical information and the level of difficulty of the content (Reference Capuano, Santo, Marsella, Molinara and SalernoCapuano et al., 2000). The customised model serves as input for the tutoring module, for which a personal user profile is useful (Al-Hanjori et al., Reference Al-Hanjori, Shaath and Abu2017; Anohina, Reference Anohina2014).
The tutoring model controls what content is taught and how, and adapts the learning process to the learner’s needs (Reference MatthewsMatthews, 2013). It uses a pedagogical knowledge model with teaching strategies to select learning material, determine its sequence, generate feedback and provide assistance (Reference AnohinaAnohina, 2014).
The user interface is the interface of the ITS that controls the interaction with the learner. It processes input, forwards it to the modules and translates the internal knowledge representation into a language that the learner can understand (Reference Al-Hanjori, Shaath and AbuAl-Hanjori et al., 2017).
The application of an ITS in design education requires extensive knowledge. The system must utilise both subject-specific and cross-domain knowledge in order to evaluate problems in context. Knowledge-based systems (KBS) are suitable for this, as they can utilise expert knowledge in a targeted manner to impart knowledge (Reference Cheok and NeeCheok & Nee, 1997; Reference FletcherFletcher, 2006). Another advantage is the separate knowledge base, which enables easy adaptation or transfer to other subject areas. In addition, KBS can separate general problem-solving behaviour from domain-specific knowledge, which enables more flexible adaptation, as changes in the knowledge base often do not require changes to the solution algorithms (Reference KurbelKurbel, 1992). The knowledge base contains facts and rules of the domain, while an inference engine monitors and controls the solution process. A user interface enables interaction and customisation of the knowledge base. A knowledge acquisition module can capture knowledge and integrate it into the knowledge base, while the explanation module makes the system’s decisions comprehensible to the user (Reference HopgoodHopgood, 2011). For example, proposed solutions from a CAD format can be automatically checked using rules and databases and recognised errors can be identified (Reference Hoppe, Gembarski, Plappert and LachmayerHoppe et al., 2020). Knowledge-based engineering systems (KBES) are considered a special form of KBS. In addition to the main components of the KBS, these also include the option of integration into a CAD system (Reference Verhagen, Bermell-Garcia, van Dijk and CurranVerhagen et al., 2012).
The specialist literature describes various systems for the digital support of design education that enable learners to test their knowledge and receive automatic feedback.
Jaakma and Kiviluoma (Reference Hopgood2019) focus on the automatic evaluation of CAD models. Their system measures the surfaces of the design and compares them with a sample solution. Incorrect surfaces are marked and reported back, but without taking into account the modelling context or individual correction notes.
Hu and Taylor (2016) analyse the ongoing modelling process by evaluating the features used and their sequence. Errors on defined solution paths can be recognised and corrective steps suggested. However, the scope of the task is very limited, as all solution paths must be defined in advance and there is no individual feedback.
Sanna et al. (Reference Hopgood201) evaluate product geometry using image recognition. This requires a well-defined template and strict model alignment, whereby only the final output is checked without taking the learning process into account.
The systems described in the literature are well suited to analysing design tasks, but are not able to support an individual learning process. They neither record the learner’s learning status nor do they generate personalised feedback. Therefore, a holistic system is developed in this article that covers all aspects of an ITS. Based on the introduction and the state of the art, the following research questions arise for the authors regarding the development of the ITS:
1) What can a model look like that can map the special requirements of design education in a digital one-to-one lesson?
2) What does a system that implements the ITS in the design education consist of?
In order to answer the research questions, the product development process based on VDI 2221 is applied, so that the four phases of planning, conceptualising, designing and elaborating are run through to develop the ITS. The following third chapter is representative of the planning phase and is used to determine the system requirements. Chapter 4 serves as the concept phase and describes the synthesis of an ITS model. The subsequent design phase is illustrated in chapter 5 by the implementation of the system in a development environment. In the final elaboration phase, chapters 6 and 7 describe the application and discussion of the ITS.
3. Requirements
To answer the questions posed, requirements are first defined on the theoretical background to enable an ITS for design education. Constructive alignment serves as a didactic framework for this, which is often used in universities to develop competency-based courses. The aim is to harmonise learning outcomes, learning activities and assessment (Reference Biggs and TangBiggs & Tang, 2007). This approach is used to derive learning and examination activities from the existing learning objectives of design engineering to formulate requirements for the ITS.
The learning objectives of design education at technical universities are comparable and include: Learning about machine elements, manufacturing processes and standards; creating sketches and drawings; modelling components and products, especially in CAD systems (Feldhusen et al., Reference Feldhusen, Brezing, Pütz, Wählisch and Boks2010; Jaakma & Kiviluoma, Reference Jaakma and Kiviluoma2019).
A central learning objective of the basic training is to enable students to independently create part and assembly models according to design rules and standards in a CAD system (Reference Plappert, Hoppe, Gembarski and LachmayerPlappert et al., 2020). The ITS is therefore specifically focussed on the CAD integration.
Practical tasks are at the centre of design education to promote application skills and illustrate the relevance of knowledge (Reference Matthiesen, Drechsler, Bruchmüller, Hildebrandt and LandhäußerMatthiesen et al., 2017). The often complex, poorly structured problems require creative thinking and support the development of metacognitive design skills (Reference Lemons, Carberry, Swan, Jarvin and RogersLemons et al., 2010). The learning system should therefore provide practical tasks that fulfil these requirements.
→ Processing of practical tasks, also with an interface to a CAD system
The advantage of one-to-one tuition is that it can be adapted to the individual needs of the learner (Reference BloomBloom, 1984). Technical learning systems can realise this strength particularly well (Reference MohammadMohammad Bagheri, 2015), as they enable clear user identification and consideration of the current learning status.
→ Automated digital recording of user-specific behaviour
Tutorial-based learning is characterised by interaction with the tutor (Reference WittwerWittwer, 2008). Here, either feedback is given on the current level of knowledge and the solutions or support is provided through recommendations on approaches and learning materials. The planned learning system should therefore offer individual feedback during the solution process and also provide personalised learning suggestions based on the current level of learning.
→ Individual feedback with visual support
In design engineering, course assessments are traditionally based on exams and project submissions, where CAD models are tested for manufacturability, strength and conformity to standards (Reference Plappert, Hoppe, Gembarski and LachmayerPlappert et al., 2020). In order for the planned system to assess learning activities, it must be able to automatically evaluate CAD models.
→ Automatic processing and analysis of CAD solutions
In engineering design, formative assessment often complements summative assessment by providing students with feedback from tutors while they are working on their assignments (Reference Jaakma and KiviluomaJaakma & Kiviluoma, 2019). In order for the digital learning system to be suitable for this, it must also be able to assess partial solutions.
→ Evaluation of partial solutions for feedback during processing
4. Conceptualisation
Models reduce complex issues to comprehensible representations (Reference VieiraVieira Kritz, 2023). For the systematic development of the ITS, a model is created that combines the ITS architecture with KBS components (see Figure 2). This model answers the first research question of what a model for design education could look like, the synthesis process of this model is described below.
Figure 2. ITS model for design education
The user interface forms a central component of the system and connects the ITS with the users and enables access to learning materials, exercises and the display of feedback and instructions.
The inference engine of the KBS, which links knowledge and makes decisions, is represented in the ITS by the learner and tutor models. The learner model processes user data as system input, while the tutor model generates system responses based on this data. Both models are essential for the interaction between the user interface and the knowledge base.
The knowledge acquisition module captures knowledge from user data and analyses the behaviour of learners, e.g. when dealing with learning materials and tasks. This data enables the system to be precisely customised to individual needs. The learner model evaluates the information obtained, checks solutions using sample solutions and restrictions, recognises errors and saves the analysed data in the knowledge base.
The tutoring model reacts to the behaviour of learners, adapts feedback individually, provides tips for finding solutions and recommends learning materials. It also generates tasks that specifically address weaknesses.
The explanatory module presents feedback and instructions in an understandable way and explains the system reactions in a way that learners can understand.
The core of every KBS is the knowledge base, which is referred to as the expert model in the ITS model. It contains the accumulated knowledge and information that the system needs to fulfil tasks, make decisions and draw conclusions. In the ITS model, the knowledge base is divided into three types of knowledge, as suggested by (Reference Capuano, Santo, Marsella, Molinara and SalernoCapuano et al., 2000) to ensure a better overview:
1) Knowledge about the learner: This dynamic knowledge component is continuously updated by the learner model. It stores the individual learning status of each learner, including general information and progress in completed learning units and tasks, as well as the errors detected there.
2) Expertise: This static knowledge component contains all application-related information on the subject area to be learnt. It also includes the knowledge required to evaluate tasks, such as cross-context knowledge, restrictions and sample solutions. In addition, alternative procedures can be saved to enhance the ITS feedback with useful hints.
3) Pedagogical knowledge: This didactic component contains the plans and procedures for communicating with the learner and aims to imitate the actions of a human teacher.
The separation of the knowledge base from the other system components has the advantage that changes can be made to the knowledge base without having to adapt the inference mechanisms in the learner model or tutor model.
The process of the ITS model is a repetitive cycle: The learner performs actions on the user interface, which are analysed by the learner model and stored in the knowledge base. This information is transmitted to the tutor model, which generates a system response based on the knowledge in the knowledge base to inform the learner. The system then waits for a new action from the learner.
5. Implementation
The ITS model is implemented in an online environment that fulfils the previously defined requirements of the model. Figure 3 shows the structure of the system and answers the second research question how a system for design education could look like. It consists of a website for data exchange and a server for data processing (see Figure 3). As the ITS is based on CAD tasks, Autodesk Inventor is integrated, as the students at the University of Hanover are already familiar with this system.
Figure 3. System environment for the ITS
To implement the user interface, a website was created using the open-source content management system WordPress. This offers several advantages: The website is easy to create, plugins and macros can be easily integrated and the learning environment can be used anywhere and at any time. Various learning materials such as texts, images, videos and interactive 3D environments can be made available on the website. Quiz environments or forms can be easily added through the integration of plugins. Feedback, hints and tasks from the ITS are also displayed on the user interface. The website also allows CAD data to be uploaded and downloaded in order to exchange tasks and solutions.
The knowledge acquisition module is responsible for individual data collection. Each learner must first register on the website, which assigns them a unique ID. The ITS collects data about the learner by recording the use of plugins, visits to certain chapters or the use of applications and stores them under the corresponding user ID. Scripts written in Python are used to connect the website to the server and enable the transfer of usage data and files.
The technical implementation of the learner model’s analysis function depends on the task format. A multiple-choice question is simply analysed by comparing the solution with the sample solution in a quiz on the website. A specially developed KBES in Autodesk Inventor is available on the server for analysing CAD models. The system evaluates the uploaded solution with the help of stored rules and data. This is equipped with VBA programming and an interface to the knowledge base. The KBES can break down the geometry of the model into surfaces and edges. At this level of abstraction of the CAD model, a detailed and robust analysis is carried out automatically by KBES. The dimensions and geometry elements recognised by the system are compared with the expert knowledge from the data memory. Corresponding deviations are evaluated as errors.
The ITS knowledge base was implemented on the server side in the form of various data stores. Static content is mapped in Excel spread sheets, while dynamic data is stored in a MySQL database. This division enables changes to the static knowledge to be made by teachers who may not have any programming skills but are familiar with Excel. A relational database is suitable for automatically populating and adapting the dynamic knowledge levels using Python scripts, as it can efficiently store and manage highly structured data in tabular form.
The ITS tutoring model was implemented with two synthesis functions on the server. The feedback synthesis, a VBA program in the KBES, reacts to the analysis of the learner model and generates feedback based on the learner’s current level of knowledge and didactic recommendations. The task synthesis creates individually customised exercises for students by automatically generating various contiguous shaft sections that contain specific errors in a CAD model with different features, such as undercuts. These tasks enable learners to find and correct errors.
The tutor model’s explanation module prepares the system output for the learner in an understandable way by utilising the website’s multimedia options. In addition to text feedback, visual feedback is provided on the CAD model which is saved as a surface model and transferred to the website. In this model faulty surfaces are displayed to the learner in color using an interactive 3D viewer plugin on the website.
6. Application scenario
To verify the model and the system environment, the application scenario is presented first. The system was developed in an initial expansion stage for gear shaft modelling. A gear shaft is used in mechanical drive or transmission systems to transmit torques and speeds. In teaching, it is an excellent example as it combines various disciplines of mechanical engineering. Modelling requires the selection of standard elements and tolerances, the consideration of production in design and dimensioning as well as mechanical design and material specifications. In addition to the practical relevance, learning objectives were also taken into account when developing the task.
To start the tutorial learning scenario with the ITS, the learner opens the website in the browser and logs in with their student email address. On the landing page, they will find multimedia learning content on gear shaft modelling, which is divided into several chapters. The learning materials include texts, images and videos that cover the basics of modelling, such as extracts from standards, case studies and explanations of manufacturing and measurement processes. In addition, the learner can use interactive 3D environments to visualise the functions of gear elements. Chapter-related quizzes are available to test knowledge and provide direct feedback on the answers.
After working through the learning materials, the learner is given a CAD exercise (see Figure 4). The task consists of modelling a gear shaft in a given installation situation and correctly designing the necessary design elements, taking into account the forces that occur. Specifications such as the gear elements to be used (e.g. bearing, gear wheel, sealing ring) and their rough arrangement as well as a maximum torsional moment are included. The system sets task restrictions, such as the use of defined standard series and maximum values for dimensions, in order to make the task assessable. Once the task has been set, the learner creates the CAD model and uploads the file to the website, where it is automatically analysed.
Figure 4. Individual input and output of the system
When learners use the website, various data is generated and stored in the system’s knowledge base (see Figure 5). When a user account is created, activities such as visits to learning chapters, quiz evaluations and the results of CAD task analyses are recorded and assigned to the learner. This data contributes to the continuous expansion of knowledge about the learner and enables a precise mapping of their current level of knowledge.
The system communicates with the learner taking into account the individual learning level and via three main components: Feedback, notes on learning materials and exercises. After uploading the CAD solution, the learner receives personalised feedback promptly. A text on the website describes the errors found, such as incorrect dimensions of standard elements, incorrect alignment diameters or a design that is not suitable for production. In addition, a 3D view of the shaft is displayed in which faulty elements are coloured. Depending on the frequency of the error, correction instructions are given that refer to standards, guidelines or further learning materials.
If the learner wants to improve their skills, they can request a practice task. This is created on the basis of their identified learning weaknesses. For example, if the learner has difficulties with the interpretation of undercuts, they will be provided with a transmission shaft with incorrect undercuts. The learner searches for these errors, corrects them in the CAD model and uploads the file for re-evaluation.
7. Discussion
The following section discusses the extent to which the ITS developed was able to fulfil the specified requirements and the extent to which it supports learners in achieving their learning objectives.
The development of the CAD task posed a particular challenge. On the one hand, the task had to be realistic and practice-orientated and, on the other, it had to remain assessable by the system with limited knowledge. Restrictions were therefore placed on the processing of the task. In order to model the gear shaft, the students had to understand the installation situation, apply guidelines and standards and take aspects of strength theory into account, as they would later in their professional lives. In particular, the correct modelling and dimensioning of standard elements such as undercuts and circlip grooves posed a challenge. These requirements for practical relevance were met, although the system is currently only designed for individual components and not for assemblies or drawing derivations - an area that still needs to be investigated further.
The technically simplest type of automatic task evaluation is the comparison of solution and sample solution, for example by means of a quiz that tests factual knowledge. When assessing CAD models, the volume and center of gravity could be compared with the sample solution. For the evaluation of partial solutions, it is necessary for the ITS to be able to recognise individual elements of the gear shaft and evaluate them in the context of the task. The verification of the learner model showed that the realisation of this requirement offers the greatest effort. A robust verification method was developed through an iterative process combined with manual solution analysis. The system is now able to recognise model elements of the shaft, classify them correctly and evaluate the dimensions in the context of the task. The requirements for automatic analysis and partial solution evaluation are therefore fulfilled, although restrictions had to be placed on the tasks due to the limited knowledge and unpredictability of solution variants.
The data for learning progress analysis is primarily obtained from the automated CAD evaluation. Error categories, such as undercut or circlip errors, are identified and stored. Additionally, plugins like quizzes and forms provide valuable tools for tracking learning progress. This data is transferred to the ITS knowledge base, enabling detailed progress assessment. The combination of KBES, plugins, and analysis tools allows for comprehensive monitoring of learning behavior. Furthermore, this approach facilitates step-by-step guidance toward solutions, similar to the Cognitive Tutor described by VanLehn (2011), though the system relies less on natural dialogue compared to the AutoTutor system by Graesser et al. (1995).
After outlining the fulfillment of ITS requirements, the discussion now turns to how an ITS can support the learning objectives of Bloom’s taxonomy levels (1984). An ITS has to accurately interpret the learner’s work. For the lower levels (Remember and Understand), a simple comparison of solution and model answer, such as through quizzes, is sufficient. To support the third level (Apply), the ITS must automate the evaluation of domain-specific solutions, such as CAD models. For the Analysis level, the ITS needs to break down solutions into components, recognise relationships, and provide learners with constructive guidance. The higher taxonomy levels (Evaluate and Create) are more challenging to support with an ITS. In the case of evaluating, individual instruction lacks the opportunities for critical reflection that group work can better facilitate. While an ITS can provide feedback to encourage self-reflection, its ability to analyse is limited due to its constrained knowledge base. The highest level (Create) is technically difficult for an ITS to address, as it relies on predefined rules and cannot identify unpredictable, creative solutions.
8. Conclusion
The contribution demonstrates how an ITS for design education can be developed by creating an ITS model to support system implementation. The system presented in this work proves suitable for implementing an ITS in design education. Notably, the interface with the CAD system Autodesk Inventor offers the advantage of allowing students to solve problems within their familiar environment. The website as a user interface provides multiple benefits: it supports multimedia presentation of information, enabling content delivery through videos or 3D environments, and visualising feedback as 3D animations. Additionally, the website facilitates extensive data collection through the integration of plugins and programs. The server ensures secure management of the growing databases, with the relational database being a particularly robust and scalable solution for automatically managing and analysing ITS data.
The implementation and application of the system demonstrate that the developed ITS model is well-suited for creating an ITS in design education. Both the analysis process for modeling the learner and the synthesis process for implementing the tutor model can be effectively represented using the KBES. The addition of the knowledge acquisition and explanation modules provides valuable methodological enhancements. These extensions ensure that appropriate tracking mechanisms are considered during system planning and emphasize the clear presentation of system output through the explanation component.
The KBES is capable of segmenting evaluations into partial solutions, enabling assessments within the context of the overall task. For example, the system can not only identify the geometry of individual model elements and evaluate their dimensions but also determine whether a particular feature is necessary at a specific shaft step and whether the chosen diameter meets the task’s load requirements.
The systems limitations primarily lie in its restricted scope of application and knowledge domain. Furthermore, the ITS requires significant development effort. Future research should explore how well the system can be adapted to other areas of design engineering and whether the existing inference mechanisms can be reused for this purpose. Additionally, alternative evaluation mechanisms, such as multi-agent systems as described by Plappert et al. (2022) or an additional combination with a graph-based analysis presented by Becker et al. (2024) could be explored to extend the scope of assessments, including with regard to assemblies, and possibly enable support for higher taxonomy levels. Implementing a dialog interface with natural language could further enhance the contextual feedback by allowing follow-up questions for deeper exploration. It is also necessary to evaluate the system in context whether it enhances personalised learning.
Open access
