1. Introduction
Intellectual Property (IP) rights, particularly patents, constitute a substantial component of modern enterprises’ intangible asset portfolios. When organisations develop novel technologies and secure patent protection, these legal instruments prevent unauthorised use while promoting further innovation and market exploitation (Reference Trappey, Trappey, Wu, Fan and LinTrappey et al., 2013). The World Intellectual Property Organization (WIPO) estimates that 90-95% of global technological innovations are documented within patent databases (Reference ChenChen, 2009). Recent data indicate that worldwide patent applications exceeded 3.55 million in 2023, representing continued growth in intellectual property activity (WIPO, 2025). Patent documents contain extensive technical information unavailable through other sources, making them valuable resources for both academic research and industrial applications (Reference Fantoni, Apreda, Dell’Orletta and MongeFantoni et al., 2013). Patent analysis provides organisations with multiple strategic benefits, including identification of technological trends, forecasting of developmental trajectories, and mitigation of infringement risks (Reference Jiang, Atherton and SorceJiang et al., 2023). However, the consistent growth in patent applications has increased the complexity of analysis while simultaneously increasing intellectual property disputes, emphasising the critical need for enhanced prior art awareness during design processes (Reference Sorce, Malizia, Jiang, Atherton and HarrisonSorce et al., 2018).
Despite their importance, designers inadequately integrate patent analysis into their product development workflows. This limitation stems primarily from patents being drafted by legal professionals using specialised terminology that makes them difficult and time-consuming to comprehend (Reference Jiang, Atherton, Sorce, Harrison and MaliziaJiang et al., 2018). Consequently, patent analysis within engineering design contexts remains a vital research area, aimed at improving designer comprehension and providing competitive intelligence while preventing intellectual property conflicts. The substantial volume and consistent growth of patent documentation make manual analysis impractical at scale (Reference Jiang, Atherton and SorceJiang et al., 2021). Critical design information is often fragmented across abstracts, background information, detailed descriptions, and claims, necessitating a resource-intensive interpretation to construct a comprehensive technical understanding.
Recent advances in artificial intelligence, particularly Large Language Models (LLMs), offer promising solutions to these persistent challenges. Models from the Generative Pre-trained Transformer (GPT) family demonstrate remarkable capabilities in processing, understanding, and generating sophisticated human language, achieving performance comparable to human experts across various benchmarks (Reference Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal, Neelakantan, Shyam, Sastry, Askell, Agarwal, Herbert-Voss, Krueger, Henighan, Child, Ramesh, Ziegler, Wu, Winter and AmodeiBrown et al., 2020). Their capacity for semantic comprehension and analysis of complex textual relationships has driven rapid adoption across specialised professional fields, from medical research to legal case analysis (Reference Zhao, Zhou, Li, Tang, Wang, Hou, Min, Zhang, Zhang, Dong, Du, Yang, Chen, Chen, Jiang, Ren, Li, Tang, Liu and WenZhao et al., 2023). Within engineering design domains, LLMs have demonstrated value in data analysis, concept generation, and design evaluation (Reference Regenwetter, Nobari and AhmedRegenwetter et al., 2022).
Large Language Models (LLMs) are increasingly transforming patent-related workflows by automating document drafting, enhancing search capabilities, and optimising analytical processes for intellectual property professionals (Reference BuiBui, 2025). Legal practices are adopting these technologies due to their ability to process, analyse, and generate high-quality patent documentation efficiently and accurately (Reference Shomee, Maity and MedyaShomee et al., 2025). As LLMs systems become embedded in design and innovation processes, the question is no longer whether they can process technical documents, but how in-depth they can understand and interpret design intent. Understanding this capacity is crucial for integrating LLMs into design reasoning and innovation strategy. Therefore, this study aims to present an exploratory investigation of commercially available LLMs’ capabilities in engineering design patent document understanding and analysis. The focus centres on three design aspects: Motivation, Novelty, and Key Invention Features, rather than statistical metrics such as patent classification numbers or citations.
2. Related work
This section reviews the intersection of patent analysis, engineering design, and artificial intelligence. The literature was identified through a systematic search of databases, including Google Scholar, Scopus and Web of Science. Keywords utilised included ‘Large Language Models’, ‘Engineering Design’, and ‘Patent Analysis’.
2.1. Patent analysis for engineering design
While patents function primarily as legal instruments for Intellectual Property protection, they simultaneously constitute a valuable repository of design case studies, providing valuable resources for innovation, education, and competitive analysis (Reference ErnstErnst, 2003). For engineering designers, the ability to systematically navigate and interpret this information is a critical competency. It provides direct insight into the state-of-the-art, prevents the redundant reinvention of existing solutions, and reveals pathways for future technological advancement (Reference Jiang, Sarica, Song, Hu and LuoJiang et al., 2022). However, superficial keyword searches or cursory document reviews are insufficient for extracting meaningful design insights, necessitating more structured and methodical analytical approaches (Reference Cascini, Fantechi and SpinicciCascini et al., 2004).
Early efforts to bridge this gap focused on creating systematic, human-driven frameworks to guide designers. These initial methodologies provided structured processes for generating novel design concepts that represent variations of existing patents while carefully avoiding infringement (Reference Hsu, Hsu, Hung and XiaoHsu et al., 2010). This line of inquiry evolved to include more sophisticated strategies for “design-around” activities, which relied on functional analysis to systematically circumvent existing intellectual property (Reference Cheng, Mi, Zhao and YangCheng et al., 2016). A key objective of this time was to develop formal representation schemes, particularly for engineering design patents, that helped designers deconstruct the working principles of an invention to better understand its core function and potential areas of conflict (Reference Jiang, Atherton, Harrison and MaliziaJiang et al., 2017).
A significant evolution in the field occurred with the shift from process-centric frameworks to more automated, computational approaches aimed at extracting deeper semantic meaning from patent text. A critical step in this direction involved creating functional representations of geometric interactions directly from patent claims, producing novel semantic annotations to identify emerging design conflicts (Reference Atherton, Jiang, Harrison and MaliziaAtherton et al., 2017). This leads the way to the broader application of Natural Language Processing (NLP) techniques. Initial integrations of NLP focused on extracting linguistic features to improve patent classification (Reference Chiarello, Cimino, Fantoni and Dell’OrlettaChiarello et al., 2018) and were extended to interpret an inventor’s underlying design intentions through the lens of established frameworks like axiomatic design (Reference Li and TateLi & Tate, 2019). Later, the development of large-scale semantic networks based on NLP started to emerge, such as TechNet, which represents engineering design knowledge derived from patent data in a structured, machine-readable format (Reference Sarica, Luo and WoodSarica et al., 2020), enabling the automated generation of design insights (Reference Jiang, Atherton and SorceJiang et al., 2021).
More recent work has focused on integrated methodologies that synthesise computational approaches with holistic design processes. Such methods combine techniques like reverse engineering and patent circumvention into multi-phase workflows that encompass problem analysis, redesign, and implementation (Reference Akerdad, Aboutajeddine and ElmajdoubiAkerdad et al., 2022). The current frontier of this research seeks to automate the identification of an invention’s core motivation and explicitly connect it to the technical specifications and structures detailed in the patent (Reference Jiang, Atherton and SorceJiang et al., 2023).
2.2. Large language models for patent analysis
The integration of Large Language Models (LLMs) in patent analysis represents an emerging and rapidly evolving field of research, opening new possibilities in areas such as novelty assessment, prior art comparison, and automated document drafting.
Initial research seeks to overcome the inherent complexity of patent language by integrating structured knowledge. Reference Zhang, Jian, Chao, Fan, Bo and ChenZhang et al. (2023), for instance, combined LLMs with knowledge graphs to enhance the semantic retrieval and classification of patent documents. This foundational work has been extended to more specialised tasks, such as multi-label hierarchical patent classification, where LLM-leveraging architectures have demonstrated substantial performance gains over previous models (Reference Rafieian and VázquezRafieian & Vázquez, 2025). The quality of input data has also been identified as a critical factor, where Reference Yoshikawa and KrestelYoshikawa and Krestel (2025) found that leveraging AI-generated summaries can significantly improve classification accuracy, particularly in challenging multilingual contexts.
Beyond analysing existing corpora, a substantial body of research has explored the generative capabilities of LLMs for creating patent content. A key development in this area is the creation of domain-specific models, such as PatentGPT, which was engineered to generate novel patent concepts and text, demonstrating superior performance on intellectual property benchmarks compared to general-purpose models (Reference Ren, Ma and LuoRen et al., 2025). This has enabled more targeted applications, including the automated drafting of technically complex patent claims. While fine-tuned LLMs can replicate correct writing styles and technical dependencies, studies emphasise that human oversight remains indispensable for ensuring legal and technical precision (Reference Chen and PanChen & Pan, 2025). This necessity for expert validation has prompted the development of Human-in-the-Loop (HITL) systems, which integrate generative AI into professional workflows to enhance efficiency while maintaining rigorous legal and ethical standards (Reference BuiBui, 2025). For less legal-sensitive tasks such as generating patent abstracts, state-of-the-art models have proven capable of producing outputs comparable in quality to those written by human experts (Reference Shomee, Maity and MedyaShomee et al., 2025).
Another popular application involves leveraging LLMs for assessing patentability, particularly novelty. Researchers have framed this application as a textual entailment problem, creating specialised datasets to benchmark model performance in this area (Reference Lee, Spangher and MaLee et al., 2024). Others have designed experiments to test if LLMs can mimic the judgment of patent examiners by comparing claims against prior art, with findings indicating that generative models can capture these complex relationships with reasonable accuracy (Reference Ikoma and MitamuraIkoma & Mitamura, 2025). To ensure these automated judgments are not “black boxes”, Reference Jang, Kim and YoonJang et al. (2023) developed an explainable AI framework that incorporates patent claim structures to provide transparent justifications for its novelty assessments. Recognising the limitations of standard text-generation metrics, Reference Yoo, Xu and CaoYoo et al. (2025) introduced PatentScore, a multi-dimensional framework that assesses the structural, legal, and semantic quality of LLM-generated claims, establishing a more reliable standard for evaluating AI-generated patent content.
Collectively, these advancements underscore a research trajectory focused on utilising the patent as a legal document. The primary objectives have been to enhance the efficiency of search, drafting, and novelty assessment from a legal perspective. However, as Reference Jiang and GoetzJiang and Goetz (2025) stated, the full potential of LLMs in the patent domain remains underexplored. The current body of work has largely overlooked the patent’s role as an artefact of engineering design. While some studies have explored tangential connections between product features and inventive value (Reference Ali and CarloAli & Carlo, 2023), a systematic exploration of engineering design patents reflecting invention motivation, technical problem framing, and the creative reasoning embodied in the design remains limited. While these advancements have transformed patent informatics, their orientation remains predominantly legal and textual. Engineering design research, on the other hand, seeks to extract functional, motivational, and conceptual insights from patents: dimensions largely overlooked by current LLM-driven studies.
2.3. Research gap and opportunity
A review of the literature reveals two parallel and disconnected streams of research. Engineering design scholarship has progressively emphasised the need for deep, conceptual analysis of patents, prioritising understanding of the core inventive rationale, specifically the motivation driving an invention and the technical novelty of its solution, as prerequisites for meaningful innovation. Conversely, Large Language Model applications in patent informatics have been overwhelmingly driven by automation of legal and textual tasks, such as prior art retrieval, document classification, and claim generation. While these applications have demonstrated significant value, their focus remains on processing patents as legal instruments rather than as sources of design insight.
A significant research gap, therefore, exists at the intersection of these fields. It remains an open question whether current LLMs, which excel at text-based pattern recognition and generation, possess the capacity for deeper inferential reasoning required to analyse patents as artefacts of engineering design. This study directly addresses this gap through an exploratory investigation that assesses the capability of modern LLMs to perform holistic design analysis of engineering design patents.
Specifically, this paper investigates the extent to which these models can: (1) identify an invention’s motivation, (2) articulate its core novelty, and (3) establish key invention features and their interrelationships. This investigation aims to determine whether LLMs can serve as effective tools for design-oriented patent analysis, potentially transforming how designers interact with patent repositories as sources of technical knowledge and innovation inspiration.
3. Research methodology
The research methodology adopted in this paper is illustrated in Figure 1. The methodology employed in this study centres on comparative analysis between LLM-generated patent analysis and expert-derived evaluations. This approach enables quantitative assessment of LLM performance in patent analysis tasks that mirror the cognitive processes employed by design engineers.
Research methodology for the comparative analysis
Figure 1 Long description
Panel A: The process begins with a patent document in PDF format. This document is reviewed by patent reviewers who conduct an individual review followed by consensus building. Panel B: The patent document is also used to generate a prompt question and an analysis example in JSON format. These are used to create an LLM prompt, which is processed by ChatGPT and Gemini. Panel C: The output from the LLM is in JSON format and is evaluated by evaluators who conduct an individual review followed by consensus building. The evaluation criteria include motivation, novelty, and key invention features, rated on a comparative scale. Panel D: The comparative rating scale includes motivation identification, novelty articulation, and inventive feature identification, each rated on a 5-level Likert scale.
To ensure consistent analytical conditions, patent documents were downloaded and stored locally, providing standardised access for both experts and LLMs. A patent analysis example was developed to maintain consistency between expert and LLM outputs, with format adaptations to suit each application: spreadsheet format for human experts and JSON structure for LLM processing. The JSON template (see Figure 2) was integrated into the LLM prompt, with the analytical outputs derived from experts on a separate patent.
To minimise potential biases while maintaining analytical feasibility, three patents spanning distinct engineering design applications were selected. The selection criteria prioritised granted US patents representing a spectrum from predominantly mechanical systems with minimal electronics to sophisticated mechatronic products:
-
• US11647866 – Grinder/Doser device for coffee beans
-
• US10631685 – Toaster
-
• US8529203 – Fan assembly
An expert evaluation was conducted by two engineering designers, each with over ten years of experience and specialised expertise in engineering design and patent analysis. Each expert independently analysed the selected patents, generating individual assessments across the three design aspects. A consensus-building approach was subsequently employed, wherein the experts engaged in structured discussions to reconcile differences and establish agreed-upon outcomes for each patent. All expert evaluations were systematically recorded in spreadsheet format to enable direct comparison with LLM-generated analyses.
The LLM analysis protocol paralleled the expert protocol. Complete patent documents were provided to the LLMs in PDF format to ensure that LLMs did not rely on internal prior knowledge or web-search hallucinations. To mitigate model-specific biases, two state-of-the-art LLMs were employed: OpenAI’s GPT-5 and Google’s Gemini 2.5 Pro.
To maintain response consistency, the LLMs were guided by a structured prompt featuring JSON output constraints. This prompt included specific instructions for deconstructing the ‘Motivation’, ‘Novelty’, and ‘Key Invention Features’ of the patents. Due to space constraints, the full prompt specification, JSON schema, and the primary patent corpus are available in an open-access repository at https://github.com/PingfeiJiang/DESIGN2026. LLM-generated JSON outputs were manually processed and transferred to spreadsheet format to enable systematic comparison with expert analysis.
While exact time-on-task was not recorded via a controlled stopwatch, the experts reported an average of 30-45 minutes per patent for full manual deconstruction and consensus building. In contrast, the LLMs (GPT-5 and Gemini 2.5 Pro) generated the structured JSON outputs in under 15 seconds per document. This represents a potential timesaving of over 90%, assuming human oversight is maintained for verification. A cross-comparison was conducted between reconciled expert results and LLM responses by the same two experts across all three patents. Evaluation employed a five-level scale applied to three analytical aspects, with standardised metrics developed for each design aspect. The evaluation framework incorporated three assessment metrics for each analytical aspect: Accuracy & Fidelity, Comprehensiveness, and Analytical Depth. Metric-specific guide questions were formulated to ensure appropriate assessment protocols (see Table 1). This systematic approach enables quantitative assessment of LLM performance relative to expert-level patent analysis across multiple dimensions. Initial independent analysis by the two experts showed an agreement rate of approximately 80%, particularly regarding factually verifiable technical features. Differences were primarily limited to the interpretative ‘Analytical Depth’ of the motivation sections, which were subsequently resolved through structured reconciliation to establish the final benchmark.
Guide questions and scaled ratings for comparative analysis between expert analysis and LLM responses
4. Results
The comparative evaluation results for GPT-5 and Gemini 2.5 Pro across all three patents are presented in Table 2. Both models demonstrated substantial proficiency in patent analysis tasks, with performance patterns revealing distinct strengths and limitations relative to expert benchmarks.
Both models achieved exceptional performance in technical feature identification, obtaining perfect scores across all analytical metrics for every patent examined. This includes the conceptually complex Bladeless Fan patent, indicating robust capability in accurately identifying and synthesising complete feature sets that constitute patent inventions. The models demonstrated particular strength in interpreting structural and functional relationships between claim elements, producing integrated system descriptions rather than just listing component parts stated in the independent claim.
Near-perfect scores were consistently achieved for Analytical Depth, demonstrating that the models perform sophisticated information synthesis rather than simple text extraction or keyword matching. This performance suggests LLMs’ capacity for technical reasoning that approaches genuine comprehension of inventive principles.
The most significant performance variation was observed in Motivation identification, where both models, particularly GPT-5, received notably lower Accuracy & Fidelity scores. GPT-5 scored 2 points for both Coffee Grinder and Bladeless Fan patents, while Gemini 2.5 Pro achieved marginally higher scores (4, 5, 3, respectively). This pattern indicates several possible phenomena: hallucination, where models generate plausible but unsubstantiated content, inferential reasoning working backwards from solutions to deduce motivations, or more broadly, extended thinking that goes beyond explicitly stated facts in the patent documents. The exact mechanism remains unclear, though the consistent deviation from source material suggests these models may be supplementing documented information with generated content that, while potentially logical, compromises factual fidelity when it diverges from inventors’ actual stated purposes.
Ratings for GPT-5 and Gemini 2.5 Pro for all three patents analysed, compared to expert analysis
LLMs’ performance remained largely consistent across patents of varying conceptual complexity. The Bladeless Fan, representing the most technically challenging patent, yielded comparable scores to simpler patents. However, subtle performance degradation was observed in Novelty analysis for the Bladeless Fan, with Accuracy & Fidelity and Comprehensiveness scores dropping to 4 and 3, respectively. This suggests that increased conceptual complexity may cause models to capture primary inventive concepts while potentially overlooking secondary or more nuanced technical details identified by human experts.
While both models performed at state-of-the-art levels, systematic differences emerged in their analytical approaches. Gemini 2.5 Pro consistently demonstrated superior fidelity to source documents, achieving higher Accuracy & Fidelity scores across multiple evaluation criteria. Conversely, both models exhibited comparable analytical depth capabilities, suggesting this represents a mature competency in current large language model architectures.
5. Discussion
The evaluation results demonstrate that contemporary LLMs exhibit substantial capability for engineering design patent analysis, particularly regarding technical feature identification and analytical synthesis. Consistently high performance in articulating novelty and interpreting interrelationships between key invention features suggests that these models transition beyond basic text processing toward a functional understanding of patented designs. This capability suggests a potential paradigm shift in engineering design practice, wherein LLMs serve as analytical partners rather than mere search or summarisation tools. Such analytical capacity could significantly accelerate critical design activities such as competitor analysis, technology landscaping, and domain knowledge acquisition. By providing rapid, coherent interpretations of complex documentation, these models allow human experts to bypass the time-intensive task of initial document comprehension and redirect cognitive resources toward strategic decision-making and creative ideation.
However, the performance degradation observed in motivation analysis represents the study’s most significant finding, revealing a fundamental limitation in current LLM approaches to patent understanding. These errors suggest a failure mode termed ‘Inferential Fabrication,’ in which models prioritise logical narrative over strict source grounding. This weakness likely stems not from insufficient capability but from the models’ default operational mode when confronted with ambiguous or sparsely documented information. Patent background and summary sections, from which motivation is typically derived, often represent the least structured portions of patent documents. When faced with incomplete information, LLMs appear to employ backward reasoning from documented solutions to infer probable motivations. This behaviour demonstrates sophisticated cognitive processing but fundamentally compromises the factual fidelity essential for rigorous patent analysis. The tendency suggests that current LLM architectures prioritise narrative coherence and logical consistency over strict adherence to source material. This operational characteristic poses a critical challenge for LLM deployment in domains that require high factual grounding, as outputs may appear plausible and well-reasoned yet contain unsubstantiated content that requires human verification.
As an exploratory investigation, this study has several limitations. The sample size of three patents, while enabling detailed quantitative analysis, precludes statistically significant conclusions about the performance of LLMs across the entirety of the engineering design domain. Additionally, the expert panel composition of two individuals represents a limited benchmark, and the incorporation of legal expertise alongside engineering perspectives could strengthen the evaluative framework. Current assessments utilise expert-assigned Likert scales, suggesting that future studies should integrate computational measures, such as semantic similarity or automated grounding checks via specific claim citations, to provide more granular accuracy metrics.
The findings suggest several promising future research directions. Large-scale validation studies employing diverse patent portfolios across multiple engineering domains and expanded expert panels would provide statistical robustness to these preliminary observations. The identified deficiencies in motivation extraction offer opportunities for targeted model refinement, particularly through fine-tuning that emphasises factual extraction over inferential reasoning. To mitigate fabrication, future protocols should implement grounding requirements that task models with citing specific paragraph or claim spans. Additionally, interactive conversational AI systems represent a compelling direction for human-in-the-loop workflows, enabling designers to pursue follow-up queries and verify claims in real-time.
From a practical standpoint, this framework enables the rapid prototyping of design knowledge. Practitioners can utilise LLM-generated JSON features to map competitor product architectures against their own during early-stage ideation. By synthesising motivation and novelty data, design teams can identify market ‘white spaces’, areas where existing intellectual property fails to address specific user needs, without the traditional friction of interpreting dense legal terminology. Our initial metrics demonstrate that LLMs can generate these structured outputs in under 15 seconds, representing a potential efficiency gain of over 90% compared to manual deconstruction. Future work will transition from this conceptual evaluation to a comprehensive workflow study to measure time-savings, decision quality, and analyst trust in professional competitive landscaping environments.
6. Conclusion
This study presents an exploratory investigation of state-of-the-art large language model capabilities in engineering design patent analysis. The evaluation framework examined three critical aspects to understand a patented invention: Motivation, Novelty, and Key Invention Features, through comparative assessment between expert analysis and LLM performance (GPT-5 and Gemini 2.5 Pro) across metrics of Accuracy & Fidelity, Comprehensiveness, and Analytical Depth. The results reveal a foundational insight into the operational logic of LLMs in this domain: they function not as passive text extractors but as active analytical synthesisers. This is evidenced by their remarkable proficiency in deconstructing a patent’s technical novelty and interpreting the complex relationships within its claims, a capability that points toward a genuine comprehension of inventive principles. This study addresses the disconnection between the prevailing legalistic applications of LLMs in patent informatics and the conceptual needs of engineering designers.
Nevertheless, a critical limitation emerged in motivation analysis, where models exhibited tendencies toward inference rather than extraction, generating plausible but potentially inaccurate content. This behaviour indicates that current LLM architectures prioritise narrative coherence over strict textual fidelity, necessitating human oversight to ensure analytical rigour and source document grounding.
Future research should address the identified limitations through larger, more diverse patent datasets and expanded expert panels to achieve statistical significance. The integration of human-AI collaborative workflows represents a promising direction for leveraging LLM analytical capabilities while maintaining the factual rigour essential for engineering design applications.
Open access
