1. Introduction
Luo and Wood (Reference Luo and Wood2017) argue based on official datasets from the U.S. patent office that the complexity of innovation processes and their outcomes has been growing steadily over the past 20 years. According to Trattner et al. (Reference Trattner, Hvam, Forza and Herbert-Hansen2019) the growing complexity of our products and systems can be seen, for example, in the increasing number of components, their interfaces, and their various ways of interaction. Futher-more, Thramboulidis (Reference Thramboulidis2008) explains that the interfaces between these components typically are not limited to one engineering discipline (e.g. mechanics, electrics, or software) anymore, but span several disciplines. Ultimately, when building modern products and systems we typically deal with a mutlitude of mechatronic components, which exhibit relevant mechanical, electrical, and software characteristics. For such mechatronic components, Youcef-Toumi (Reference Youcef-Toumi1996) already identified that they come with their very own integration issues, which are not considered in the classical engineering disciplines.
In the past, the sole ability to engineer such complex systems might have been a suffcient success factor for societies and organizations across the globe. As Senescu et al. (Reference Senescu, Haymaker, Meža and Fischer2014) argue, today due to rising global competition in many domains the engineering ability alone is not sufficient anymore, but the efficiency and effectiveness of the underlying engineering processes must be taken into account. According to Zidane and Olsson (Reference Zidane and Olsson2017), different definitions exist for the terms efficiency and effectiveness in this context. One possible definition of efficiency is that the overall engineering process can be executed as fast as possbile and the desired target system’s time-to-market is minimized. In contrast, one possible definition of effiectiveness is that the engineering process leading to the desired target system consumes as little resources as possible and, hence, the overall process costs are minimized. As Strauss (Reference Strauss1985) already argued, one key to efficiency (i.e. high development speeds) is dividing the overall work into individual activities, which can be executed in parallel, but also might require synchronization of their work results. Hartmann and Briskorn (Reference Hartmann and Briskorn2022) explain that these activities must be scheduled carefully such that the work results of one activity are ready when they are needed by another acitivity, leading to the well-known project scheduling problem and the critical path. When required work results are not ready in time, you typically have two choices: Either you delay follow-up activities potentially causing a suboptimal utilization of project resources as Kumaraswamy and Chan (Reference Kumaraswamy and Chan1998) explain for the construction industry. Or you start follow-up activities anyways and risk working with wrong assumptions causing unnecessary revisions of work results later in the process as Chapman (Reference Chapman2006) argues from the viewpoint of project and risk management.
In contrast, Maalem and Zarour (Reference Maalem and Zarour2016) emphasize that one key to effectiveness is to understand stake-holder requirements and validate assumptions as early as possible in the engineering process. According to Mobin et al. (Reference Mobin, Li, Cheraghi and Wu2019) validation activities must be planned where applicable to ensure that such mistakes are uncovered and resolved quickly and to minimize their impact on the future course of the project. In this context, agile methodologies and frameworks have emerged over the past two decades ensuring stakeholder integration along the entire development process as Chan and Thong (Reference Chan and Thong2009) describe. In agile approaches, according to Deininger et al. (Reference Deininger, Daly, Lee, Seifert and Sienko2019) stakeholders have the obligation to review work results frequently and give their feedback to engineering team.
Furthermore, Hackenberg et al. (Reference Hackenberg, Zehetner and Frühwirth2023) argue that online collaboration platforms with vision management, issue tracking, and task scheduling support engineering teams in achieving high degrees of efficiency and effectiveness. Tony Liu and William Xu (Reference Tony Liu and William Xu2001) already explained that the idea behind such web-based platforms is to bundle all relevant project information in a central repository, to which all stakeholders have access from anywhere, anytime, and with any device. And Kraemer and King (Reference Kraemer and King1988) already noted that online collaboration platforms can improve the speed of information flow between the stakeholders significantly compared to traditional approaches. Higher speeds of information flow, in turn, reduce the risk of project delays as well as incorrect assumptions and unnecessary revisions of work results as Novak and Sellnow (Reference Novak and Sellnow2009) show, e.g., in the food industry.
Now, as academic teachers we need to prepare our students as good as possible for what they will face in practice. Ideally, our graduates quickly find their place in multi-disciplinary engineering teams and contribute reliably to the success of their projects. However, beyond the core technical skills they need a solid understanding of the overall engineering processes, the overarching efficiency and effectiveness objectives, as well as the employed collaboration principles as Deshpande and Huang (Reference Deshpande and Huang2011) argue. According to Meyer and Norman (Reference Meyer and Norman2020) teaching such profound understanding of process, team, and collaboration dynamics still remains a major challenge for engineering education.
We believe that one major obstacle to stimulating a solid understanding in these “soft” areas is the complexity of commercial-grade (CAD) software and their rather steep learning curves. Various industries require this level of complexity for their specific applications, but mastering this complexity not always represents the learning goals of academic programmes and courses.
Research question
To overcome the current situation in engineering education, we wanted to understand how well differ-ent course formats work based on feature-limited online collaboration platforms and the digital LEGO framework. In particular, we wanted to understand the suitability of different course formats as well as the collaboration platform and the digital LEGO framework for Master-level product design and sys-tems engineering education. Furthermore, we were interested in how well such course formats work for children from high school aged between 10 and 15 years old, how well they can deal with CAD-like programs, and how well they understand the inherent collaboration challenges.
Contribution
To answer the previous research questions, we conducted two Master-level courses on product design and systems engineering as well as one workshop with high school children. The organizational structure of the two Master-level courses as well as the workshop, and the work results of the different teams are explained in Section 2. During and after conducting the experiments we collected feedback from the individual participants and analyzed the data as discussed in Section 3. Finally, we draw our conclusions from the previous experiments and their data, derive our key learnings, and sketch out important future work towards high impact engineering education in Section 4.
2. Course experiments
For our experiements we used the open source software CADdrive Footnote 1 as the underlying online collabora-tion platform with basic version management, issue tracking, and task scheduling functionality, as well as the open source software LeoCAD Footnote 2 for digital LEGO design based on the open LDraw Footnote 3 data for-mat and parts library. Subsequently, we first describe our Master-level course formats that we conducted with students from University of Applied Sciences Upper Austria in Section 2.1, before explaining our workshop format with that we conducted high school children in Section 2.2. Both for the Master-level courses and the workshop we introduce the organizational settings and show the most interesting work results delivered by the individual teams.
2.1. Master level
At Master level, we conducted two independent courses in two different study programmes with slightly varied study plans and, hence, organizational settings. Both courses spanned an entire semester, required about 30 work hours per student, and included grading based on project participation as well as quantity and quality of project deliverables. The goal of the first course was to design a energy-optimal glasshouse for growing sweet tomatos over the winter months in Alpine regions as explained in Section 2.1.1. Furthermore, the course included a fair amount of simulation tasks for verification and validation. The goal of the second course was to design a paper folding machine turning plain paper sheets into boxes as expained in Section 2.1.2. The second course did not include simulation tasks, but verification and validation was achieved through manual reviews in this case.
2.1.1. Glasshouse for Alpine regions
The first course started in the winter term 2023 in the full-time Master programme “Innovation and Prod-uct Management”. The participants included 5 women and 5 men between 26 and 40 years old having between 1 and 16 years of work experience in different industries (e.g. automotive, retail, banking, etc.) and departments (e.g. product development, logistics, martketng and sales, etc.). Furthermore, 8 students were holding a Bachelor degree and 2 students were holding a Master degree in some field. Also, most students were already familiar with some CAD tool (e.g. PTC Creo, AutoDesk AutoCAD, Siemens NX, etc.) as well as simulation software (e.g. Abaqus, Matlab Simulink, Ansys, etc.).
In this project-based course, the goal of the group was to design an energy-effcient glasshouse for grow-ing sweet tomatos over the winter months in Alpine regions. The tasks included requirements analysis, modular architecture design, mechanical design, electrical design, and simulation-based verification of selected system properties. Figure 1 shows the mechanical design developed by the group with digital LEGO bricks and LeoCAD. The mechanical design comprises a grow area for the plants with glass walls and windows as well as solar panels on the roof (see Figure 1-b), and a shed area for storing tools and batteries (see Figure 1-c).
Figure 1. Mechanical design of the glasshouse with digital LEGO bricks
In the context of simulation-based verification, the group was tasked with applying finite element analysis (FEA), computational fluid dynamics (CFD), and multi-physics simulation (MPS) for analyzing selected system properties. The group decided to apply finite element analysis for analyzing the stability of the mechanical structure under snow and wind pressure, as well as computational fluid dynamics for analyzing the heat loss characteristics of the building. Furthermore, the group decided to employ multi-physics simulation with OpenModelica Footnote 4 for analyzing the electrical energy consumption caused by the automatic window opening and closing system as shown in Figure 2.
Figure 2. Multi-physics simulation of automatic window opening and closing system
2.1.2. Paper folding machines
The second course also started in the winter term 2023 in the part-time Master programme “Robotic Systems Engineering”. The participants included 1 woman and 12 men between 25 and 33 years old having between 3 and 10 years of work experience in different industries (e.g. logistics, chemical, agri-culture, etc.) and departmets (e.g. automation engineering, business development, project management, etc.). Furthermore, all students were holding a Bachelor degree in some technical field. Also, most stu-dents were familar with some CAD software (e.g. AutoDesk AutoCAD, Dassault Systèmes Catia, PTC Creo, etc.), while only two students did not have prior CAD experience.
The course was divided into two parts, a theory part followed by a practice part. In the theory part differ-ent concepts such as systems engineering, systems lifecycle processes, requirements analysis, systems architecture, systems integration, as well as verification and validation were introduced. In the practice part the students were split into teams and were given the goal of developing a paper folding machine, which turns plain paper sheets into boxes with defined measures. The tasks included requirements anal-ysis, mechatronic systems architecture derivation, as well as mechanical, electrical, and logical system design. Figure 3 shows three different mechanical designs proposed by the teams.
Figure 3. Different mechanical designs of the paper folding machine
In contrast, Figure 4 shows the electrical design and software design of one paper folding machine. This particular team worked with EPLAN Electric P8 Footnote 5 for the electrical design as well as Excalidraw Footnote 6 for the software design of the target system. Note that in this case one student of the team was already familiar with EPLAN Electric P8 and had access to the tool through his employer, which is why the tool was selected. Furhtermore, note that the notation for the software design was not prescribed by the course instructor, but resembles some form of state chart formalism, which is common in control engineering. Finally, as described previously the validation of the different designs was performed with manual review techniques across all teams.
Figure 4. Electrical and logical design of one paper folding machine
2.2. High school level
At high school level, we conducted one single-day workshop with children from around Wels city in Upper Austria. The workshop was organized as part of the KinderUni Footnote 7 summer programme at the the School of Engineering of the University of Applied Sciences of Upper Austria in Wels, Austria. The KinderUni programme provides children the chance to experience science hands-on and to learn about different topics such as engineering, biology, architecture, and many more. Section 2.2.1 explains the structure of this single-day workshop and highlights results developed by the children during the course of the workshop.
2.2.1. LEGO product ideas
The workshop took place in July 2024 and was organized by the Upper Austrian branch of the KinderUni in Wels city Footnote 8 . The workshop participants included 7 girls and 12 boys between 10 and 15 years old from different primary and high schools in the region, some – but not all – knowing each other from the same school. In the morning, the children were brought to the School of Engineering of the University of Applied Sciences Upper Austria by their parents and were welcomed by an official representative of KinderUni. The workshop itself lasted from morning until late afternoon and included short breaks before and after noon as well as a lunch break.
During the workshop, the hosts first presented the workshop agenda and the goal of the design projects before explaining and demonstrating the basics of LeoCAD for digital LEGO design. Then the children were divided into teams of two with one girl working alone due to the uneven number of workshop participants. The goal of each team was to develop an idea for a new LEGO product, while the target group (e.g. male or female), the target age (e.g. under 10, under 16, over 18, etc.), and the target theme (e.g. science fiction, princess, holiday, etc.) for the product could be selected freely. Figure 5 shows three product designs, which have been proposed by the teams.
Figure 5. LEGO product designs proposed by the teams during the workshop
For all teams, one requirement was that they uploaded the current state of their digital LEGO designs every hour to CADdrive, so that the design process can be reconstructed and visualized. For teams including two members, another challenge was dividing their work into two independent modules, devel-oping the two modules separately, and integrating the modules at the end of the workshop day. Three out of nine two-member teams managed to master this challenge (as, e.g., shown in Figure 6), while three two-member teams decided not to divide their work, but decided to work together in front of one computer only. The remaining three two-member teams failed integrating their work.
Figure 6. Components of the space station LEGO product idea and their assembly
3. Evaluation results
After conducting the experiments from the previous section, we performed a systematic evaluation on the Master-level courses as well as an informal feedback round on the workshop to understand the current quality of our educational offering. For systematic evaluation, we developed a questionnaire assessing the prior experience of the course participants in different knowledge domains as well as the learning experience during the course. Note that there were some differences between the questionnaires of the two Master-level courses, which is why not all quantitative results add up to the total number of participants (i.e. 24 students). In the following, we first present the results of the assessment of prior expierience in Section 3.1 before going into the details of the assessment of the learning experience in Section 3.2.
3.1. Prior experience
Figure 7 summarizes the results of the assessment of prior experiences of our Master-level course par-ticipants. In our questionnaire we assessed prior experiences in different fields such as CAD/CAM (medium), simulation (i.e. FEM/CFD/MBS/MPS/DES; low), PDM/PLM (high), QFD/DSM (low), product management/design (medium), systems/requirements/mechanical/electrical/software engineer-ing (high), as well as project management, agile methodology, and design thinking (high). Note that in each category the participants selected between novice, junior, professional, senior, and expert experience levels, which we aggregated to low, medium, and high levels here for gaining a bigger picture.
Figure 7. Summary of prior experience in different knowledge areas
3.2. Learning experience
For assessing the learning experience during the two Master-level courses on product design and systems engineering, the questionnaire included a quantitative and a qualitative section. In the quantitative section we assessed the general learning experience, the learning experience in different knowledge areas (e.g. digital collaboration and agile methodology), and the learning experience with the different software tools (e.g. CADdrive and LeoCAD) as described in Section 3.2.1. In the qualitative section we asked the participants about ideas for improvement of the two different course concepts as well as the software tools employed as described in Section 3.2.2.
3.2.1. Quantitative feedback
Figure 8 summarizes the results of the assessment of the general learning experience during the courses. In this section of the questionnaire, we assessed the overall experience, the study programme relevance, the study programme integration, the course concept, the course organization, the motivation level, the activation level, the LEGO framework, and the use case. As one can see, the tendency among all factors is positive with overall experience, study programme relevance, course concept, and use case rated among the best. On the other hand, one can see that the study programme integration, the course organization, and the digital LEGO framework show potential for improvement.
Figure 8. Summary of general learning experience during the course
Then, Figure 9 summarizes the results of the assessment of the learning experience in different knowl-edge areas. In particular, we assessed the learning experience the areas such as project management, agile methodology, product design, requirements engineering, computer-aided design, computer simu-lation, digital collaboration, and interdisciplinary collaboration. Again the learning experience shows a positive tendency in all knowledge areas with product design, requirements engineering, and digital col-laboration being among the best. On the other hand, one can see potential for improvement in areas such as project management, agile methodology, computer-aided design, and systems engineering.
Figure 9. Summary of learning experience in different knowledge areas
Finally, Figure 10 summarizes the results of the assessment of the learning experience with different software tools. In particular, the section of the questionnaire included the suitability, the usability, and the stability of the software tools CADdrive as the underlying collaboration platform, LeoCAD as the digital LEGO design tool, SimScale as a tool for finite element analysis and computational fluid dynamics, and OpenModelica as a tool for multi-physics simulation. The results show a positive tendency for most software tools and characteristics including CADdrive at the forefront. However, the results also reveal usability and stability issues with LeoCAD, which hinder the learning experience.
Figure 10. Summary of learning experience with the different software tools
3.2.2. Qualitative feedback
The qualitative feedback section included three open questions about improvements of the course con-cept, improvement of the software tools, and miscellaneous feedback. The feedback about the course concept mainly included requests for providing better course material with the course (e.g. about the theory of agile methodology and digital collaboration) as well as setting clear goals and communicat-ing expectations for the outcomes of the product design and systems engineering projects. The feedback about the software tools mainly included requests for CAD and simulation tool training prior to the course (e.g. multi-physics simulation with OpenModelica) as well as means for automatic up-scaling the brick resolution of LEGO models. The LEGO brick resolution of the product and system models turned out to become an issue as the mechanical design teams progressed and started to add details, which were negelected initially. Finally, the miscellaneous feedback mainly repeated the previous requests and included some positive statements about our innovative course format.
4. Conclusions
In this paper we first highlighted the need for innovation in engineering education with focus on stim-ulating a profound understanding of interdisciplinary process, team, and collaboration dynamics. Then, we presented our experiences with two Master-level courses on product design and systems engineering as well as a one-day workshop with children from high school including quantitative and qualitative par-ticipant feedback. In the following, we summarize our most important learnings from the experiments and sketch possible directions of future work in this field of research, development, and innovation.
Learnings
While you possibly can learn many different things from the conducted experiments, here we concentrate on those learnings which we believe to be most generalizable and impactful for the product design and systems engineering community. We decided to number the individual learnings for future reference and to enable more efficient and effective discussios around the underlying principles and mechanisms of design and engineering education. With each learning we include a short discussion of possible threats to validity, which are inherent with any scientific research study and the employed research methods.
Learning 1 Overall, the two courses on product design (with simulation-based verification) and systems engineering (with mechanical, electrical, and logical design as well as review-based validation) using online feature-limited collaboration tools such as CADdrive and digital LEGO design tools such as LeoCAD seem to be well suited for training Master students with different educational and professional backgrounds in a number of knowledge areas. Most prominently, the courses seem to foster learning in areas such as digital and interdisciplinary collaboration (i.e. team work) as well as product design and requirements engineering with relatively high levels of student motivation and activation.
Learning 2 Most Master students rated the digital LEGO framework as a viable foundation for such courses and learning experiences, but the opinions seem to be more controversial on this topic. When looking closer at the evaluation results, one can observe that the participants of the product design course (i.e. the glasshouse case) consistently preferred the digital LEGO framework, while the partici-pants of the systems engineering course (i.e. the paper folding machine case) preferred using classical CAD instead. Three possible explanations are that the two cases are not equally well suited for the LEGO framework, that the systems engineering group faced more issues with LeoCAD due to model complexity, and that the systems engineering group had stronger prior CAD knowledge.
Learning 3 The evaluation results also show that both the overall learning experience as well as the learning experience in the different knowledge areas still can be improved. When looking at the overall learning experience, the organization of the two courses seems to be the most effective lever for improv-ing the course ratings. This impression can be confirmed when looking closer at the qualitative feedback showing mutliple requests for better course material as well as clearer definition of goals and expec-tations. We expect that additional course material with relevant theoretical inputs probably also helps improving the learning experience in the respective knowledge areas (e.g. agile methodology).
Learning 4 Finally, the work results as well as the informal feedback of the KunderUni workshop showed us, that our workshop format based on online collaboration platforms such as CADdrive and the digital LEGO framework are well suited for inspiring high school children for design and engineering topics. However, due to time limitations the workshop only covered digital LEGO design as well as version management (mastered by all teams) and component integration (mastered by three out of nine teams). We are confident, that with more time and proper training/tooling we can also cover topics such as requirements analysis, issue management, task scheduling, and simulation-based verification.
Outlook
To advance the field of product design and systems engineering education even further, we currently work on several topics: First we work on better CAD support for the digital LEGO framework to avoid the usability and stability issues experienced during our experiments. Second we work on integrating classical mechanical and electrical CAD tools and their models into CADdrive to improve the learning experience of groups with stronger prior CAD knowledge. And third, we work on adding explicit sup-port for requirements management, systems architecture (including decomposition and interfacing), and computer simulation to improve the learning experience in these domains as well.
Beyond that we plan to extend the platform capabilities towards mechatronic product line engineering as explained by Michalek et al. (Reference Michalek, Ebbes, Adigüzel, Feinberg and Papalambros2011). We believe that proper training concepts for product line engineer-ing represent the next frontier, that we need to master in academic education so that our companies can benefit from these skills in the future and build larger varieties of products more efficiently and effec-tively. Consequently, we need to add concepts such as product platforms, module repositories, interface compatibilities, and configuration models to CADdrive. Furthermore, we need to redesign our courses to cover these topics in the course material as well as the cases considered during practical exercises.
Open access
