1. Introduction
Well-educated engineers are crucial for successful engineering design (Reference Albers, Denkena, Matthiesen, Albers, Denkena and MatthiesenAlbers et al., 2012; Reference Lindemann, Birkhofer and GrabowskiLindemann & Birkhofer, 1998). Therefore, engineering design related courses are part of the mechanical engineering curriculum (Reference Albers, Denkena, Matthiesen, Albers, Denkena and MatthiesenAlbers et al., 2012; Reference Dillenhöfer, Kossack, Sersch, Künne, Bender and GustDillenhöfer et al., 2025). Fundamental courses for engineering design, e.g. technical drawing, are often attended by a large group of students up to 500 participants (Reference DillenhöferDillenhöfer, 2023; Reference Metraglia, Baronio and VillaMetraglia et al., 2011). These students have differing levels of knowledge and experience in engineering design education caused by their various secondary education backgrounds, vocational trainings or completed pre-engineering courses (Reference Kannengiesser, Gero, Wells and LammiKannengiesser et al., 2015; Reference Kossack and BenderKossack & Bender, 2022; Reference Žeželj and MilerŽeželj & Miler, 2018). Lecturers in these fundamental engineering design courses mostly use frontal learning activities like lectures (Reference Bender, Husung, Kirchner, Kletzing, Kossack, Lohrengel, Mayar, Riedel and StahlBender et al., 2025; Reference DillenhöferDillenhöfer, 2023). So, all students are taught the same content at the same pace, even though they start at different levels of knowledge and competence.
Due to the high number of participants, there is almost no interaction between the teachers and the students, and the group size inhibit students to ask questions (Reference Eckert, Seifried and SpinathPfäffli, 2015). Individual support to compensate the existing heterogeneity would require large tutoring capacities (Reference PfäffliEckert et al., 2015), but supplementary learning material has been shown to be quite useful in the engineering field for heterogeneous student groups (Reference Žeželj and MilerŽeželj & Miler, 2018). Several e-learning contents for engineering design education automatically provide students with feedback, e.g., Reference Becker, Hoppe, Plappert, Herrmann, Gembarski and LachmayerBecker et al. (2025); Reference DillenhöferDillenhöfer (2023); Reference NobesNobes (2025). However, it is difficult for students to use the feedback and identify suitable material or e-learning content appropriate to their individual background and context, and review available content on their own, especially during the transition from high school to college in the first-year courses (Reference Arnold, Kilian, Thillosen and ZimmerArnold et al., 2018).
One approach to deliver personalized learning materials to students without creating immense workloads for tutors and lecturers is the concept of adaptive e-learning (Reference KerrKerr, 2016). This can involve both individual and personal needs and preferences (Reference ReyRey, 2009; Reference van Seters, Ossevoort, Tramper and Goedhartvan Seters et al., 2012). An adaptive e-learning environment has been developed for a first-year engineering design education course (Reference Kossack and BenderKossack & Bender, 2023). It is implemented in the learning management system Moodle version 4.3 (Moodle Contributors, 2025) and considers the adaptivity criteria knowledge and competencies. An initial evaluation of the adaptive e-learning environment resulted in positive feedback from the students. According to the students, the environment helps to structure the self-study time, gives important and interesting background information for the frontal lectures and motivates users to work on the content in the self-study time. The exam results of one exam indicate a positive impact of the use for acquiring professional competencies. In addition, using the adaptive e-learning environment can reduce knowledge advantages based on before-university experiences, e.g. internships, vocational trainings, specific technical school education and other practical experience. (Reference KossackKossack, 2025)
E-learning content comes with a high initial development effort, but after that, a large group of students can use this content without additional or with less expense for the tutor or lecturer (Reference SchönwaldSchönwald, 2007). Therefore, this paper analyzes the long-term integration and impact of such an e-learning content with the main research question:
• What is the impact of offering an adaptive e-learning environment in engineering design education?
Interesting points are (1) To what extent do students use the adaptive e-learning environment? and (2) To what extent does the use support acquiring professional competencies?
To answer the research question, Section 2 details the specific adaptive e-learning environment for engineering design education AdE-Le. Section 3 starts with the data collection and analysis method to gain the results described thereafter. Finally (in Section 4), the results are discussed, and further research demand is derived from the findings.
2. The adaptive e-learning environment for engineering design education (AdE-Le)
AdE-Le is an adaptive e-learning environment developed at the Ruhr-University Bochum, Germany. The environment identifies students’ existing competencies with questions, provides students with detailed feedback for their answers, an interpretation of their learning level and displays appropriate content, e.g., short videos or digital content for their individual learning level. The use of AdE-Le is recommended in self-study time and helps to organize it and choose suitable learning content for individual student needs.
Reference KossackKossack (2025) describes the didactical development of AdE-Le in detail. It is based on the steps of the instructional design (in accordance with Reference Niegemann and WeinbergerNiegemann and Weinberger (2019)) and includes the typical steps of the development of e-learning content (in accordance with Reference Arnold, Kilian, Thillosen and ZimmerArnold et al (2018)). The development includes the main phases: analysis, development and implementation.
The analysis phase for AdE-Le confirmed the general needs of such individual support and led to specification of detailed requirements. It identified, e.g., the widely used learning management system Moodle (Moodle Contributors, 2025) as a suitable implementation tool. The phase also determined the topic of AdE-Le. It addresses typical content in first year engineering degree programs, for which practical technical experience is a helpful prerequisite for acquiring competencies: dimensioning on technical drawings, tolerancing and the calculation of fittings (Reference KossackKossack, 2025).
A key step in the development phase is the connection of content to user behavior based on the Constructive Alignment (in accordance with Reference Biggs, Tang, Chan and PawlinaBiggs and Tang (2020)). The development was carried out by refining learning outcomes into automatically assessable tasks, implementing the tasks and connecting appropriate feedback and suggested learning content to the given answers (Reference Kossack and BenderKossack & Bender, 2023).
To implement AdE-Le, the Moodle function Activity Lesson is used. This enables the combination of content pages with pages for branching with questions. On the content pages are short videos, interactive content, or links to books chapters and DIN standards. Pages for branching with questions mostly use closed question types like Single-Choice or matching tasks. Specific feedback is prepared and stored for each individual answer option. For each page created, jumps to following pages are defined depending on the user’s behavior. So, the navigation in AdE-Le is based on a decision tree that links the content to chosen answers in these questions.
Figure 1 shows an exemplary simplified learning path implemented with Activity Lesson. The branching question asks students to choose a suitable second view for the complete dimensioning of the item. In the illustrated case, the student chose “view C”. Depending on the chosen answer, AdE-Le shows the item in the matching 2D view and asks if the dimensioning is correct for that view. If the student agrees and chooses “Yes” in this case, they need to repeat dimensioning rules and get them displayed. If students know that the presented dimensioning is wrong and select “No”, AdE-Le offers spatial ability training. After both contents, students are offered another try.
Exemplary simplified learning path of AdE-Le in an activity lesson about dimensioning based on Reference KossackKossack (2025)
Figure 2 illustrates the stored feedback on a question about using tolerance tables. Students get advice depending on their answer. The developed answer options are based on typical mistakes that were anticipated by the lecturer, e.g., choosing a wrong tolerance class or using the wrong nominal dimension.
Simplified single-choice question in an activity lesson (left) with stored feedback (right) based on Reference KossackKossack (2025)
Assessment limitations in Activity Lesson led to the use of Activity Quiz. Activity Quiz offers the possibility that questions are randomly selected and therefore students receive different questions when they repeat the activity (Moodle Contributors, 2025). Furthermore, STACK (STACK Contributors, 2024) questions can be used to store mathematical formulas and thus check multi-step calculations with a decision tree-based assessment. To this purpose, lecturers anticipate mistakes and implement these with several nodes. As a result, subsequent mistakes are assessed as in the exam tasks and detailed feedback is available on what exactly the students have done wrong, e.g., feedback if they used a not suitable formula or didn’t change the unit correctly. The use of STACK questions in AdE-Le is similar to the use in our automatically assessable exam tasks on calculating machine elements, as detailed in Reference Kossack, Uttich and BenderKossack et al. (2025). The Activity Quiz can store detailed feedback for every question and answer. Students see this feedback after submitting their results. Defined prerequisites for the activities and completion criteria connect the activities to one learning environment. Figure 3 illustrates the connection of selected activities. Three more Activities Lesson (Fitting Basics, Tolerances and Advanced Dimensioning) are available, if a student finishes the last page of the Activity Lesson Dimensioning Basics. Finishing the Lesson Fitting Basics is mandatory to start the Activity Test Fitting Calculations. This Activity Test is completed if a student reaches at least 70% of the points. In case of failing the test, repetitions are possible, but with varying tasks. In case of failure, students get extra content and recommendations in addition to the detailed feedback of every task.
Extract of AdE-Le: connection of selected activities simplified from Reference KossackKossack (2025)
Figure 3 Long description
The flowchart illustrates the structure of an e-learning course on dimensioning basics. The course starts with Dimensioning Basics, which branches into three activities: Fitting Basics, Tolerances, and Advanced Dimensioning. Each of these activities leads to a Fitting Calculations step. If the student scores 70 percent or higher, they proceed to the next stage. If not, they receive feedback for every question and content recommendations. The feedback includes access to additional learning materials and videos. The flowchart also includes labels for Moodle Activities, which are Activity Test and Activity Lesson.
3. Impact of using the adaptive e-learning environment AdE-Le
3.1. Data collection and analysis
Students at Ruhr-University Bochum have had access to AdE-Le in the course Engineering Design A since the winter term 22/23. The course is a first-year course for the degree programs Mechanical Engineering and Sales Engineering and Product Management and offered every winter term. The course consists of 50% presence-study time and 50% self-study time. The presence-study time is mainly frontal in lectures with up to 300 participants. In addition, tutors support students when the students work on exercises and answer their questions.
For the self-study time, the lecturers recommend different content: books, exercises with exemplary solutions and AdE-Le. A lecturer introduced AdE-Le in the frontal lecture each year and explained the functions in detail. The demonstration contained the way AdE-Le supports the purposeful use of self-study time, other benefits from using it, and the research ideas. To encourage students to use AdE-Le, bonus points were added to the exam points of a student for completing all AdE-Le activities. There was no gradation for the time or the number of attempts a student needed to complete a learning unit.
AdE-Le was available to all participants of the Moodle course belonging to the Engineering design course. Lecturers didn’t change any content in the course or in AdE-Le in the last three years, since the underlying norms and standards weren’t changed during that time. But lecturers adjusted the scheduling of the different activities of AdE-Le in accordance with the lecture dates.
The final summative exam of the course consists of tasks addressing spatial imagination and different views including axonometric projection, dimensioning and the calculation of fittings. This means the exam includes tasks with intended learning outcomes not addressed by AdE-Le (views) and intended learning outcomes addressed by AdE-Le (dimensioning and calculation of fittings). The three tasks mentioned are rated differently in the exam. Students can score twice as many points with the tasks about views than with the task about dimensioning. Students can participate in the exam about two months after the course. If they don’t want to or fail their first attempt, they can repeat the exam five or eight months after the course. Every year, the exam has the same structure: it starts with the tasks about views and ends with the task on dimensioning and a fitting calculation.
We investigate the impact of using digital learning content for the acquisition of competencies by analyzing the exam results of the different tasks without counting any bonus points for participation. In doing so, we only take exams results of first year students into account, as the year of studying is the most significant factor influencing the exam results in engineering design (see our previous study in Reference Kossack, Kattwinkel and BenderKossack et al. (2024)). Moreover, we only select data sets of the first offered exam (two months after the course), because participating in the subsequent course Engineering Design B in the ongoing degree program at the time of (re-)taking the exam in Engineering Design A might influence the results.
For statistical analyses, we divide the exam data sets into two test groups. The groups are (1) students who used AdE-Le and (2) students who did not use it. We examine the null hypothesis, which assumes, that there is no real difference between users of AdE-Le and non-users with the Mann-Whitney-U-Test (Reference Döring and BortzDöring & Bortz, 2016). The rejection range is defined by the significance level, which is usually set at 5 % (Reference HollenbergHollenberg, 2016). In case of identified significance, Cohen’s effect size d is considered. A d up to 0,3 is a small effect, up to 0,5 a medium sized effect and greater 0,5 a large effect (Reference CohenCohen, 2013). The tool for quantitative analysis was SPSS for Windows (version 31). In the absence of a proper control group to test this hypothesis, the results of other exam tasks about views are used as a comparison.
3.2. Results
Table 1 shows the number of users of AdE-Le. It differentiates between students who started the first activity and students who finished all activities. In these data, the overall number of users is significantly higher in the first year of offering AdE-Le (22/23) and roughly halves with each subsequent year. And this even though the number of overall exam participants does not decrease. The percentage of students who finished all activities is rather similar for the three analyzed years. It is equal to about half of the number of students who started the usage of AdE-Le.
Number of AdE-Le users and number of exam data sets to consider
Because of the declining number of AdE-Le users, the number of data sets from first-year students is rather low for the two subsequent years (23/24 and 24/25) with only 27 and 19 user data sets. With the rather constant number of overall data sets, the test group is clearly smaller than the test group of non-users. Figure 4 compares the percentage of exam task points of the two test groups (users and non-users) for three different exam tasks of the three analyzed years.
Initial analysis of the exam results for the three years 22/23, 23/24 and 24/25 for the two test groups with learning outcomes addressed by AdE-Le Dimensioning (D), Fitting calculation (F) and the comparison tasks about Views (V) divided into data sets of users of AdE-Le and non-users
Figure 4 Long description
The image contains three separate bar graphs representing exam results for the years 22/23, 23/24, and 24/25. Each graph compares the performance of users and non-users of AdE-Le in three tasks: Dimensioning (D), Fitting calculation (F), and View tasks (V). The vertical axis represents exam points in percent, ranging from 0 to 100 percent. The horizontal axis lists the tasks D, F, and V. Users are represented by diamonds and non-users by triangles. Each data point includes error bars indicating variability. Panel A (Year 22/23) shows users scoring higher in Dimensioning and Fitting calculation tasks compared to non-users, with both groups performing similarly in View tasks. Panel B (Year 23/24) indicates improved scores for both users and non-users across all tasks, with users consistently outperforming non-users. Panel C (Year 24/25) shows further improvement, with users achieving the highest scores in all tasks and non-users also showing significant progress.
Figure 4 shows the mean value with standard deviation for each task and test group. In all data sets, the users of AdE-Le reach more points in the tasks than the non-users. However, this difference is considerably greater for the tasks with learning outcomes addressed by AdE-Le (dimensioning task D and fitting calculation task F) than for the task with learning outcomes not addressed by AdE-Le (views V). The detailed statistical analysis (see Table 2) supports that finding. In the year 22/23, the differences between the test groups are statistically significant for the tasks about Dimensioning and Fitting calculation with a medium or nearly medium effect (see Cohens’ d) and not statistically significant (see the significance p >0.05) for the task about views. The data sets from winter term 23/24 show a statistically significant difference between the test groups for all three tasks. However, the effect size is considerably greater for the tasks with learning outcomes addressed by Ade-Le (dimensioning and fitting calculation) than for the task with learning outcomes not addressed by AdE-Le (views). The analysis of the data sets from winter term 24/25 displays a statistical significance between the two test groups for all tasks with a medium effect size. However, even in this data set, the smallest effect size is for the task about views, which learning outcomes are not addressed by AdE-Le.
The differences between users and non-users in the comparison task about views shows that in the years with the smaller number of users (year 23/24 and year 24/25), only the high performing students used AdE-Le.
Test results for statistical significance with the test groups users (n1) and non-users (n2)
The resulting points for all tasks vary each year regardless of grouping data sets into test groups. The students in 23/24 reached in mean about 10% more points in all the tasks than the students in 22/23. The data sets of 24/25 are overall similar to the ones in 23/24. However, the results in the different tasks vary. The mean value of points in the tasks assessing views was highest in 24/25 and the mean value of points in the task assessing fitting calculation was highest 23/24.
4. Discussion and outlook
Students use AdE-Le, however the number of users and also the percentage of participants in the course decreased in the years after the development and initial evaluation. This decline is not based on the introduction to the students or the incentive for participation in the form of bonus points on the exam results, because it has been the same over the three years. The only difference is a feedback questionnaire, available after finishing all activities, that was only included in winter term 22/23. The difference between the students who started to use AdE-Le and those who finished AdE-Le is rather constant and corresponds to the overall discontinuation rate of the degree programs. However, feedback from the students would have been helpful to those students who started but did not complete AdE-Le. It could have helped to get ideas for improving AdE-Le or to understand why an adaptive e-learning environment does not meet the needs of these students. Reusing the feedback questionnaire might also get ideas for improvement from students who overall liked AdE-Le or only finished it because of the extra exam points. Therefore, two feedback questionnaires are planned for the use in winter term 25/26. One questionnaire targets students, who do not use AdE-Le or do not finish it. The other contains detailed questions about the different activities for students who used it.
The analysis of the exam results shows a difference between the users and the non-users. This difference also exists in exam tasks with assessed learning outcomes not addressed by AdE-Le. Probably AdE-Le users are more interested in the topic, have a higher motivation or better learning competencies, which leads to the use of AdE-Le but also overall better exam results. However, the differences between the test groups are greater in all three years in the tasks assessing learning outcomes addressed by AdE-Le. This allows for the conclusion that the use of AdE-Le has a positive impact of acquiring professional competencies in engineering design education.
The existing difference between the data sets from the three years shows the importance of encouraging a high number of students to use AdE-Le. In the comparison task about views the rather small user groups of AdE-Le in the year 23/24 and the year 24/25 have significantly better results. This means only the high performing students used AdE-Le. Whereas in the year 22/23 is no significant difference between the users of AdE-Le and non-users in the comparison task about views, so there is no significant performance difference between the users of AdE-Le and non-users. However, in the tasks with learning outcomes addressed by AdE-Le is a significant difference of the two test groups in the year 22/23. This indicates that the use of AdE-Le has a high impact especially for this group of students on the exam results. In the other two years, when only the high performing students used AdE-Le the use only enlarged the existing difference of the test groups.
The exam data shows that even with the use of AdE-Le, the learning outcomes about dimensioning and fitting calculations are not fully reached. One reason might be the order of the tasks, because the tasks with learning outcomes addressed by AdE-Le are the last ones and students normally start with the first task. We assume that in case of time pressure, students do not work on the latter tasks from our observation, that many students did not work on the task about dimensioning (see median of non-users years 22/23). Consequently, the exam results do not indicate the level of reaching the learning outcomes about this topic. Another reason might be the alignment of the exam format to the teaching and learning activities including the content in AdE-Le and the exam tasks. Maybe other assessment tasks or methods, e.g. for long-term competence acquisition, would indicate deviating impacts of using AdE-Le.
Grouping the students for the analysis is subject to several limitations. The group of users is not a random sample, as seen in the tasks about views: The users of AdE-Le are the higher performing students in the course. Students who only used some activities of AdE-Le are in the group of non-users. The experiment design included offering bonus points on the exam for finishing AdE-Le, which may lead students to finish an activity without using its content purposefully. For testing on statistical significance, the test groups in the data sets 23/24 and 24/25 differ in size and in 24/25 also in standard deviation for the tasks with learning outcomes addressed by AdE-Le. This might lead to inaccurate test results.
The data shows general differences in the exam results over the three years. While most students in 22/23 used AdE-Le, the results in tasks about dimensioning and fittings were, in overall mean, not the best of the three years. However, the comparison task about views has the worst mean value. The course had the same teaching and learning activities and the same lecturer with no obvious changes in lecture sheets or exercises nor in AdE-Le over the three years. However, the explanations in the lectures vary and especially address typical mistakes of the last exams. The number of lectures varies because of holidays, and the exam tasks address the same learning outcomes, but vary in the exact tasks. And students might have different prerequisites due to the Covid-pandemic. Other courses also have 10% of mean value changes in the exam results, so this might be normal fluctuations.
The results show the benefit of using an adaptive e-learning environment with difference in the exam results between users of an adaptive e-learning environment and non-users. But they also clarify the importance of encouraging a large group of students to use it, as especially the impact for students not performing high in the comparison task seems to be helpful. For further investigation, the feedback and exam results of more users, maybe also from different universities, would be helpful to understand the impact of offering such an e-learning environment. In addition, an evaluation should include the question of whether the use of the AdE-Le influences other teaching and learning activities. We assume, e.g., that students might not participate in lectures, because they use the adaptive e-learning environment.
5. Conclusion
The paper analyses the impact of offering the Moodle-based adaptive e-learning environment AdE-Le. AdE-Le includes content about dimensioning, tolerance and fittings and is developed for self-study time in addition to existing teaching and learning activities like lectures. The data sets in three years for an engineering design course show that the number of AdE-Le users is declining. The analysis of the exam results shows a positive correlation between the use of AdE-Le and the exam points reached. However, using AdE-Le influences the overall exam results more, if a wider range and number of students use it and not only the high performing students. This highlights the importance of encouraging students to use offered e-learning content.
Acknowledgement
The initial development of an adaptive e-learning environment for engineering design education for first-year students is sponsored by a digi-Fellow of the DIGITALE HOCHSCHULE NRW (2022-2023).
Open access
