2.1. Methodological design of the survey

To answer the research questions, a standardized online survey was designed, filled out by a group of engineers, and analyzed. Table 1 provides an overview of the questionnaire. The core focus is the collection and analysis of example free text prompts used to search for specific assemblies. Prior to designing the survey, five expert interviews were conducted with representative users from the target group. These interviews aimed to identify practical perspectives with professional relevance for the survey. Additionally, domain-specific terminology was collected to ensure that the survey would be correctly understood by participants. A key finding from the interviews, which informed the survey design, is that all respondents indicated they primarily search for assemblies based on their function and the assembly-specific parameters that determine functional performance. Examples of such parameters include main geometric dimensions, applied forces, and component lead times.

Table 1.

Survey on prompt-based assembly retrieval

The interviews further revealed that classifying these parameters into geometric dimensions, physical properties, and organizational constraints is considered useful. This qualitative insight forms a component of the evaluation framework for the quantitative survey and justifies the formulation of hypothesis H1, which will be validated or refuted through analysis. The interviews also helped specify response options for multiple-choice (MC) items. In addition to MC questions, the survey includes single-choice (SC) items and open text questions. The latter type is crucial for providing engineers with high flexibility in formulating prompts. The questions are organized into four categories. In the first part of the survey, two questions on personal information capture the participants’ demographic (D1) and professional (D2) profiles. This is followed by a multiple-choice (MC) question on different types of search information (S-I) for assemblies, aiming to identify participants’ preferences and priorities for prompt formulation. The following four search scenarios for prompt-based assembly searches (P1–P4) form the core of the survey. Three assemblies relevant to the design work of the user group were predefined through the interviews to ensure that the proposed search scenarios are pertinent to the participants’ daily tasks and that the engineers possess the expertise to select the most important assembly attributes and parameters for prompt formulation. To obtain quantitatively reliable results on search behavior, the content-related context from which the search is initiated must necessarily be standardized. To provide this standardized context for the hypothetical search scenarios, all participants are presented with the same image of the assembly and the same instructions for prompt formulation. In question P4, participants are asked to create a prompt for an assembly from their own design work to compensate this bias introduced by the standardized prompting context in questions P1–P3. Questions E1–E3 conclude the survey with an exploratory section, aiming to gather additional insights and opinions from participants on prompt-based assembly searches. Participants are asked about the expected advantages and disadvantages of a prompt-based search approach compared to existing methods, as well as further suggestions for improving assembly search in general.

2.2. Collection of prompt formulations for assembly search

Based on the interviews, three assemblies from the field of handling technology were selected for the search scenarios to generate prompts. In consultation with the interviewees, the representativeness of the assemblies and the participants’ expertise with these assemblies were confirmed. The assemblies are visualized in Figure 1 and are briefly described below in order of increasing complexity. Example prompts are also given in order to provide an idea of the participants’ prompt formulation.

The first assembly (a) is a pneumatic lifting unit with two lateral guide rods, as well as pneumatic and geometric interfaces for energy supply and connection to adjacent components. The second assembly (b) is a double pneumatic parallel gripper with the grippers arranged at a 90° angle. The entire gripping unit can be rotated 180° around the vertical axis via a pneumatic swivel mechanism and mounted to a robot arm using a console. The third assembly (c) is a handling unit composed of various pneumatic components. A parallel gripper (blue) is attached to a swivel unit (pink), which is in turn connected to another swivel unit (brown). This assembly is mounted on a pneumatic linear unit (light blue), which is fixed to a stand with screw connections to the surrounding structure. Overall, handling is achieved through a combined motion of two rotations and one translation of the gripper.

In addition to the prompts for these three predefined assemblies, prompts for participants’ individual assemblies (P4) were also collected. The textual assembly prompts are analyzed on the basis of six prompt characteristics whose evaluation allows the research questions (RQ) Q1 and Q2 to be addressed. Table 2 provides an overview of the prompt characteristics and their mapping to the research questions. The prompt characteristics are divided into general properties and assembly-specific properties. General properties are analyzed collectively for all prompts, as they are independent of the prompted assembly. Specific prompt properties depend on the respective prompted assemblies and are therefore analyzed individually for each search scenario. There are four general prompt characteristics (G1–G4) and two specific prompt characteristics (S1–S2). Regarding the syntactic formulation of prompts, characteristic G1 measures the prompt length in whole characters. Characteristic G2 focuses on the syntactic structure, distinguishing whether the prompt is written as bullet points or as continuous text. Since the interviews emphasized that functional descriptions of assemblies play a central role in prompting, characteristic G3 therefore captures whether and how the functional description is expressed linguistically. The assembly function can be explicitly described in text or inferred implicitly, for example from the assembly name. In some cases, however, it is not possible to infer the function from the prompt. With respect to the informational content of prompts, characteristic G4 records the number of specified assembly parameters as a whole number. Characteristic S1 relates to the naming of the assembly and allows observation of different naming variations.

Figure 1.

Assemblies used for search scenarios in the survey and example prompts

Characteristic S2 addresses the geometric, physical, and organizational parameters used to specify the assembly. The specific characteristics S1 and S2 form the core of the analysis, as they provide a quantitative answer to the first research question. The values of the general and specific characteristics are extracted individually for each prompt to enable analysis of the prompts and inferences about the prompting behavior of the user group.

Table 2.

List of prompt properties for analyzing engineers’ assembly retrieval behavior

3.1. Structure of the participant group

The survey was conducted with the company Bosch Manufacturing Solutions and participant group consisted of designers in mechanical special purpose machinery development. Due to the specific design requirements, the technical products developed are predominantly custom machinery rather than standardized series products. A total of 48 engineers participated in the survey. Participants’ professional experience in mechanical design (D1) was recorded in years. The average design experience is 14.5 years, with a median of 11.5 years. The lower quartile ranges from a minimum of one year to eight years of experience, while the upper quartile ranges from 20.5 to a maximum of 35 years. The participant group therefore includes engineers across all experience levels. The professional profile of the participants (D2) was captured through their roles and responsibilities. Fifty percent of the engineers are responsible for the design of individual products, 27% hold project responsibility for an entire customer order, and 6% occupy management positions in mechanical development. Another 6% are students, and the remaining 11% hold other roles, such as standardization or internal process management. To protect company interests, no further details regarding the composition of the participant group are disclosed in this publication.

3.2. Preferred assembly search information

For question S-I on preferred information when searching existing assemblies for reuse, participants primarily identified the qualitative description of the assembly function in continuous text as their preference. Overall, 83% of respondents considered this relevant for assembly searches. A share of 72% of participants wished to use specifications of the motion sequences performed by an assembly in their search. Similarly, engineers considered the description of energy supply (71%) and applied mechanical loads (67%) to be relevant for assembly searches. Organizational properties of an assembly, such as the cycle time of the implemented process (52%), reference projects in which a suitable assembly is sought (43%), and information on purchased components included in the assembly (37.5%), were also deemed relevant by many participants. The installation space of the assembly was cited as a relevant search criterion by 37.5% of respondents. Notably, connection geometries to adjacent assemblies (12.5%) and the assembly mass (10%) were considered relevant search criteria by only a few participants. An overview is shown in Figure 2. Overall, these results confirm the insights from the interviews and hypothesis H1. The core of assembly searches should be a qualitative textual functional description, supplemented by physical, geometrical, and organizational parameters of the target assembly.

Figure 2.

Preferred search information for assembly retrieval

3.3. Insights into prompting mechanical assemblies for reuse

A total of 169 prompts for assembly searches were created by the 48 participating engineers. There are 47 prompts for the lifting unit, 46 for the gripper system, 44 for the handling assembly, and 32 for other assemblies. The general prompt characteristics are presented first, followed by the specific prompt characteristics.

3.3.1. General prompt properties

The four general prompt characteristics G1–G4 were analyzed collectively for all 169 assembly prompts. An overview is shown in Figure 3. Characteristic G1 measures the prompt length in characters. The majority of prompts (83%) are shorter than 100 characters, with an average length of 66.1 characters. This can be explained by the distribution of characteristic G2: A predominant share of 92% of prompts is written as bullet points, while only 8% are formulated in complete sentences. Consequently, most prompts are syntactically structured primarily as a sequence of the assembly name and specifying assembly parameters. Regarding the type of functional description in G3, it is notable that none of the prompts contains an explicit textual description of the assembly’s functional performance. This initially appears to contradict the assessment of the importance of functional descriptions from question S-I. However, for most assemblies (84%), the function is conveyed indirectly through a semantically meaningful naming of the assembly in the prompt.

Figure 3.

General prompt property distribution

The use of bullet-point formatting in G2 may explain why functional relationships within the assembly, as well as the overall function, are not explicitly stated, since functions are typically expressed as noun-verb constructions (Reference Bender and GerickeBender & Gericke, 2021), and individual items of assembly information cannot be sufficiently related in a functional sense without complete sentences. Consequently, the assembly is described in most prompts through its parameters. Characteristic G4 shows that prompts contain an average of 2.7 assembly parameters, with most prompts including three or more parameters. The highest number of parameters used to describe assemblies in the search scenarios was seven.

3.3.2. Assembly-specific prompt properties

In this chapter, the prompts for the individual search scenarios are analyzed separately to draw conclusions about participants’ behavior in naming assemblies and the assembly parameters used. An examination of assembly naming is followed by an analysis of the parameters used for assembly specification. An overview of assembly naming for P1–P3 is provided in Figure 4. For the linear unit (P1.S1), the term “Lifting Unit” is used in the majority of the 47 prompts (55%). Other common synonyms are “LiftPos” (17%) and “Lifter” (4%), as well as “Linear Unit” (4%). However, 12% of prompts contain no recognizable name. The terms “Cylinder Guide,” “Hubix,” and “Pn Cylinder” are each used in a single prompt. It is evident that terms are not used consistently by participants, and prompt-based search systems will require flexible interpretation of the submitted search terms if no standardized nomenclature for assembly names exists within the company. It can be assumed that this effect would be even more pronounced in real search scenarios, as participants were already provided with suggested terms in the instructions for P1–P3. The same observation is evident in the analysis of the 46 prompts for the gripper (P2.S1). 30% of respondents adopted the term “Double-Parallel Gripper” from the instructions for P2. Other common names are “Double Gripper” (20%), “Gripper” (15%), and “Parallel Gripper” (13%). The terms are very similar and merely provide implicit descriptions of different aspects of functional performance. This may be related to the fact that different functional properties have varying relevance for the individual work of the engineers. No identifiable name could be determined in 11% of the prompts. The terms “Double-Jaw Gripper,” “Swivel Gripper,” “Gripper Head,” “Robot Gripper,” and “Change Gripper” are each used in a single prompt. For the 44 prompts regarding the handling assembly (P3.S1), it is notable that most prompts (32%) contain no name. Beyond this, the term “Pick & Place” (20%) is the most widely used. The terms “Handling Unit”, which stems from the instructions, and “Pivot Unit” are both used in only 11% of prompts. It is also notable that the functional understanding of assemblies overlaps: The handling assembly is referred to both as a “Gripper” (11%) and as a “Linear Unit” (6%). While both are subsystems within the assembly, these terms do not describe its core function. Other terms used include “Motion Unit” in two prompts and “Positioning Assembly” in one prompt. The remaining 32 prompts for other assemblies cover a highly diverse range of modules. Therefore, analyzing synonyms provides little insight. However, other observations can be made. Both “Lifting Unit” and “Handling System” appear in three prompts each, making them the most frequently mentioned. This suggests that the example assemblies were chosen to be representative of the participants’ design work for the survey. It is also notable that the names similarly convey functional details of the assembly in an implicit manner. Other examples of assemblies include “Frame,” “Clamping Device,” and “Palletizer,” each of which also appear in three prompts.

Figure 4.

Distribution of assembly names for P1 - P3

For prompt characteristic S2, which relates to assembly parameters, the previously predicted distribution of the most relevant parameters from question S-I is largely confirmed, in contrast to the relevance of functional descriptions. In particular, assembly motions, applied mechanical loads, and energy supply are frequently used as parameters. Physical and geometric parameters dominate the assembly description, although organizational parameters are also included. Compared to S-I, there are qualitative differences in the organizational parameters: no reference projects are mentioned, but operational properties, such as cycle time or suppliers of purchased components, are occasionally cited. The distribution of parameters for each assembly is shown in Figure 5. The most important parameters, included in more than 15% of prompts, are briefly described below. For the lifting unit (P1.S2), the lifting stroke is specified in 77% of prompts. The lifting force is mentioned in 68%, and energy supply is defined as pneumatic in 66% of prompts. A typical prompt therefore reads: “Pneumatic Lifting Unit, 250N, 120mm stroke.” Prompts for the gripper assembly (P2.S2) frequently include the properties “Gripper Force” (65%), “Gripper Stroke” (46%), and “Gripper Arrangement” (26%). Additionally, the gripper dimensions (15%) and energy supply (15%) are specified in many prompts. For the handling assembly (P3.S2), it becomes evident that assembly parameters play a more dominant role in identification compared with the assembly naming (P3.S1). The assembly often has no name but contains more parameters in the prompts than the other assemblies. It can therefore be assumed that parameters have a more significant influence on the search process from the user perspective. Most frequently, the search is based on the actuation strokes of individual components (80%). Actuation forces of the components (36%), their motion sequences (34%), and energy supply (25%) are also commonly specified. Additionally, the relative coordinates of individual movements (20%) and the overall dimensions of the assembly (20%) are occasionally included in the prompts.

Figure 5.

Distribution of assembly parameters for P1 - P3

Figure 5 Long description

Panel P1.S2: A bar graph depicts assembly parameters for a lifting unit. The horizontal axis represents different parameters, and the vertical axis represents the number of prompts. Parameters include lifting stroke, lifting force, energy supply, guiding property, operating process, assembly dimensions, manufacturer, and workpiece dimensions. The bars are color-coded into three categories: physical parameter in green, organizational parameter in orange, and geometrical parameter in blue. Lifting stroke has the highest number of prompts, followed by lifting force and energy supply. Panel P2.S2: A bar graph depicts assembly parameters for a gripper. The horizontal axis represents different parameters, and the vertical axis represents the number of prompts. Parameters include gripper force, gripper stroke, gripper arrangement, dimensions, energy supply, functional features, mass, operating process, and manufacturer. The bars are color-coded into three categories: physical parameter in green, organizational parameter in orange, and geometrical parameter in blue. Gripper force has the highest number of prompts, followed by gripper stroke and gripper arrangement. Panel P3.S2: A bar graph depicts assembly parameters for a handling unit. The horizontal axis represents different parameters, and the vertical axis represents the number of prompts. Parameters include actuation strokes, actuation forces, direction of motion, energy supply, operating process, coordinates, dimensions, mass, functional features, and manufacturer. The bars are color-coded into three categories: physical parameter in green, organizational parameter in orange, and geometrical parameter in blue. Actuation strokes have the highest number of prompts, followed by actuation forces and direction of motion.

For the 32 prompts on other assemblies (P4.S2), more general statements about the search parameters can be derived due to the diversity of the described assemblies. Interestingly, the geometric dimensions of the assemblies are included in only 10%–20% of prompts for P1–P3, but in 50% of the prompts for other assemblies. This suggests that geometric dimensions generally play a more significant role than in the examples from P1–P3. Otherwise, searches for the other assemblies also predominantly rely on assembly motions (32%), applied mechanical forces (32%), and energy supply (27%).

3.4. Participants’ opinion on prompt-based assembly search

The assessments of the surveyed engineers indicate that prompt-based search methods in mechanical engineering offer several potential advantages (E1) from a user perspective. According to participants’ subjective perceptions, the use of natural language inputs can simplify and accelerate the search process, resulting in reduced workload and time savings. It was particularly emphasized that free-text searches are not constrained by predefined search masks or formal input structures, and this cognitive relief enables more flexible, individualized, and intuitive formulations. This is considered beneficial for more efficient handling of complex information. At the same time, respondents also expressed concerns (E2) regarding the reliability and traceability of results from prompt-based searches. Doubts were raised about how accurately the returned results reflect the actual search intentions and how their validity can be systematically verified. Additionally, participants highlighted that the quality of the underlying data is a central prerequisite for the success of such systems. Incorrect or incomplete classification of historical assemblies cannot be compensated for by a prompt-based approach. Furthermore, it was noted that effective use of prompt techniques requires training, making targeted instruction necessary.

Overall, the engineers’ assessments suggest that prompt-based search approaches in mechanical engineering are perceived as promising but not without limitations. The perceived benefits largely depend on the quality of existing assembly data, the reliability of the search, and the experience and methodological competence of the users regarding the interaction with LLMs.

The participant group also evaluated various measures for improving assembly search (E3). The creation of an ontological assembly structure based on core functions within the company was considered particularly useful (64.6%). This functional classification is intended to provide the basis for a more targeted and semantically consistent prompt-based search for specific assembly types. The integration of existing search mechanisms from legacy systems was also supported by a large proportion of respondents (62.5%) for the purpose of transferring prior experience and established structures into new search systems. Consolidating search functions into a single tool also received broad approval (54.2%), as this could prevent media discontinuities and standardize search processes. There was also support (54.2%) for the possibility of uploading image sources such as technical sketches or photos – in addition to text inputs – so as to enable multimodal search. Furthermore, half of the participants (50%) considered improving search speed a key factor in enhancing the practical usability of future systems.

These results indicate that respondents do not see the development of assembly search exclusively in the context of prompt technology usage, but they regard prompt-based approaches as central component within a search system architecture featuring comprehensive functional and technical coordination.