2.1. TWA – A virtual hub for digital chemistry

TWA is a pioneering project that creates a universal digital twin of the real world, building on the early potential of the Semantic Web to enhance cheminformatics and broader chemical applications (Berners-Lee et al., Reference Berners-Lee, Hendler and Lassila2001; Taylor et al., Reference Taylor, Gledhill, Essex, Frey, Harris and De Roure2006; Murray-Rust, Reference Murray-Rust2008). The Semantic Web, an evolution of the World Wide Web, addresses the gap between human-readable documents and machine-readable data by prioritizing semantics over presentation. It builds on layers of technologies standardized by the World Wide Web Consortium, starting with raw data expressed using Unicode and uniquely identified through Internationalized Resource Identifiers (IRIs). Information is represented through the Resource Description Framework (RDF) in triples of subject, predicate, and object. Ontologies formalize this knowledge by defining the structure of instances within a KG, enabling querying via SPARQL. The Semantic Web’s core principle is Linked Data, which promotes accessibility and integration across different sources.

Initially conceptualized in 2010, TWA has evolved from the representation of a single chemical industry park on Jurong Island (Singapore) into an unrestricted world model capable of integrating a range of phenomena from the atom to multiscale features impacting environment, climate, and population health (Akroyd et al., Reference Akroyd, Mosbach, Bhave and Kraft2021), including power and heat network optimizations for CO₂ savings, environmental monitoring, and cross-domain climate resilience planning through the Climate Resilience Demonstrator (Mosbach et al., Reference Mosbach, Menon, Farazi, Krdzavac, Zhou, Akroyd and Kraft2020; Akroyd et al., Reference Akroyd, Mosbach, Bhave and Kraft2021; Akroyd et al., Reference Akroyd, Bhave, Brownbridge, Christou, Hillman, Hofmeister, Kraft, Lai, Lee, Mosbach, Nurkowski and Parry2022). TWA operates on the principles of the Semantic Web and adheres to the FAIR guidelines, to ensure that all data are findable, accessible, interoperable, and reusable (Wilkinson et al., Reference Wilkinson, Dumontier, Aalbersberg, Appleton, Axton, Baak, Blomberg, Boiten, da Silva Santos, Bourne, Bouwman, Brookes, Clark, Crosas, Dillo, Dumon, Edmunds, Evelo, Finkers, Gonzalez-Beltran, Gray, Groth, Goble, Grethe, Heringa, ‘t Hoen, Hooft, Kuhn, Kok, Kok, Lusher, Martone, Mons, Packer, Persson, Rocca-Serra, Roos, van Schaik, Sansone, Schultes, Sengstag, Slater, Strawn, Swertz, Thompson, van der Lei, van Mulligen, Velterop, Waagmeester, Wittenburg, Wolstencroft, Zhao and Mons2016). It integrates software agents that manage information flows, interface with computational models, and continuously enhance TWA’s KGs with new data (Zhou et al., Reference Zhou, Eibeck, Lim, Krdzavac and Kraft2019; Akroyd et al., Reference Akroyd, Mosbach, Bhave and Kraft2021).

The digital chemistry in TWA is aligned and structured around foundational ontologies such as OntoSpecies, OntoKin, OntoCompChem, and OntoPESScan, facilitating a comprehensive mapping of chemical species, reaction mechanisms, and quantum chemistry calculations, respectively (Kondinski et al., Reference Kondinski, Mosbach, Akroyd, Breeson, Tan, Rihm, Bai and Kraft2024a, Kondinski et al., Reference Kondinski, Bai, Mosbach, Akroyd and Kraft2023). This framework supports detailed data relationships and enhances interoperability, enabling multifaceted data usage and reducing ambiguities (Farazi et al., Reference Farazi, Krdzavac, Akroyd, Mosbach, Menon, Nurkowski and Kraft2020; Akroyd et al., Reference Akroyd, Mosbach, Bhave and Kraft2021). Additionally, computational agents in TWA perform complex tasks such as calibrating kinetic mechanisms and automating discovery processes (Kondinski et al., Reference Kondinski, Mosbach, Akroyd, Breeson, Tan, Rihm, Bai and Kraft2024a), exemplified by the development of novel MOPs (Kondinski et al., Reference Kondinski, Menon, Nurkowski, Farazi, Mosbach, Akroyd and Kraft2022) which, among a variety of applications, can be used for photocatalytic CO₂ reduction (Ghosh et al., Reference Ghosh, Legrand, Rajapaksha, Craig, Sassoye, Balázs, Farrusseng, Furukawa, Canivet and Wisser2022; Adeola et al., Reference Adeola, Ighalo, Kyesmen and Nomngongo2024).

The OntoMOPs ontology is designed to provide and enrich semantic relationships between MOPs, chemical building units (CBUs), and assembly models (Kondinski et al., Reference Kondinski, Menon, Nurkowski, Farazi, Mosbach, Akroyd and Kraft2022). This ontology enables advanced query capabilities for professionals engaged in the modeling and preparation of MOPs, supporting informed decision-making with detailed information on the construction and functionalities of these materials. OntoMOPs links manually curated MOP instances to crucial metadata such as molecular mass, charge, formulas, and provenance information like DOIs and CCDC numbers for precise identification and cross-referencing with crystalline databases. Additionally, the assembly model concept details how different generic building units (GBUs) contribute to the formation of specific polyhedral shapes recognized in reticular chemistry, such as tetrahedra and octahedra, while the CBU concept models chemical functionalities and binding sites necessary for MOP formation.

2.2. Trends in knowledge-intensive chemistry QA systems

In recent years, the field of NLP has experienced a remarkable rise in popularity, primarily driven by the accessible deployment of LLMs. The advent of LLMs is marked by their ability to tackle diverse knowledge-intensive tasks that range from the humanities to the sciences, including chemistry (OpenAI et al., Reference Achiam, Adler, Agarwal, Ahmad, Akkaya, Aleman, Almeida, Altenschmidt, Altman, Anadkat, Avila, Babuschkin, Balaji, Balcom, Baltescu, Bao, Bavarian, Belgum, Bello, Berdine, Bernadett-Shapiro, Berner, Bogdonoff, Boiko, Boyd, Brakman, Brockman, Brooks, Brundage, Button, Cai, Campbell, Cann, Carey, Carlson, Carmichael, Chan, Chang, Chantzis, Chen, Chen, Chen, Chen, Chen, Chess, Cho, Chu, Chung, Cummings, Currier, Dai, Decareaux, Degry, Deutsch, Deville, Dhar, Dohan, Dowling, Dunning, Ecoffet, Eleti, Eloundou, Farhi, Fedus, Felix, Fishman, Forte, Fulford, Gao, Georges, Gibson, Goel, Gogineni, Goh, Gontijo-Lopes, Gordon, Grafstein, Gray, Greene, Gross, Gu, Guo, Hallacy, Han, Harris, He, Heaton, Heidecke, Hesse, Hickey, Hickey, Hoeschele, Houghton, Hsu, Hu, Hu, Huizinga, Jain, Jain, Jang, Jiang, Jiang, Jin, Jin, Jomoto, Jonn, Jun, Kaftan, Kaiser, Kamali, Kanitscheider, Keskar, Khan, Kilpatrick, Kim, Kim, Kim, Kirchner, Kiros, Knight, Kokotajlo, Kondraciuk, Kondrich, Konstantinidis, Kosic, Krueger, Kuo, Lampe, Lan, Lee, Leike, Leung, Levy, Li, Lim, Lin, Lin, Litwin, Lopez, Lowe, Lue, Makanju, Malfacini, Manning, Markov, Markovski, Martin, Mayer, Mayne, McGrew, McKinney, McLeavey, McMillan, McNeil, Medina, Mehta, Menick, Metz, Mishchenko, Mishkin, Monaco, Morikawa, Mossing, Mu, Murati, Murk, Mély, Nair, Nakano, Nayak, Neelakantan, Ngo, Noh, Ouyang, O’Keefe, Pachocki, Paino, Palermo, Pantuliano, Parascandolo, Parish, Parparita, Passos, Pavlov, Peng, Perelman, Peres, Petrov, Pinto, Michael, Pokrass, Pong, Powell, Power, Power, Proehl, Puri, Radford, Rae, Ramesh, Raymond, Real, Rimbach, Ross, Rotsted, Roussez, Ryder, Saltarelli, Sanders, Santurkar, Sastry, Schmidt, Schnurr, Schulman, Selsam, Sheppard, Sherbakov, Shieh, Shoker, Shyam, Sidor, Sigler, Simens, Sitkin, Slama, Sohl, Sokolowsky, Song, Staudacher, Such, Summers, Sutskever, Tang, Tezak, Thompson, Tillet, Tootoonchian, Tseng, Tuggle, Turley, Tworek, Uribe, Vallone, Vijayvergiya, Voss, Wainwright, Wang, Wang, Wang, Ward, Wei, Weinmann, Welihinda, Welinder, Weng, Weng, Wiethoff, Willner, Winter, Wolrich, Wong, Workman, Wu, Wu, Wu, Xiao, Xu, Yoo, Yu, Yuan, Zaremba, Zellers, Zhang, Zhang, Zhao, Zheng, Zhuang, Zhuk and Zoph2023). However, despite their impressive performance on standardized examinations, general-purposed LLMs like GPT-4 often struggle with more advanced and specialized requests, revealing their lack of in-depth understanding of the subject matter (Guo et al., Reference Guo, Guo, Nan, Liang, Guo, Chawla, Wiest and Zhang2024). While fine-tuning is a possible remedy (Zhang et al., Reference Zhang, Liu, Tan, Chen, Yan, Yan, Li, Huang, Yue, Ouyang, Zhou, Zhang, Su, Zhong and Li2024), a significant challenge remains: these models are inherently limited by the scope and recency of their training data, rendering them inadequate for querying up-to-date information or applying the latest research knowledge without undergoing further retraining.

In-context few-shot learning is a technique where LLMs are provided with a few input–output examples during testing to align their behavior with user expectations without updating their weights. In-context means the model processes and utilizes examples provided within the same input context, rather than relying on weight updates or external training. First observed in GPT-3 during model scaling experiments (Brown et al., Reference Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal, Neelakantan, Shyam, Sastry, Askell, Agarwal, Herbert-Voss, Krueger, Henighan, Child, Ramesh, Ziegler, Wu, Winter, Hesse, Chen, Sigler, Litwin, Gray, Chess, Clark, Berner, McCandlish, Radford, Sutskever and Amodei2020), in-context learning enables models to perform tasks based on demonstrations alone, and recent research suggests that smaller models can also be trained for this capability (Min et al., Reference Min, Lewis, Zettlemoyer, Hajishirzi, Carpuat, de Marneffe and Meza Ruiz2022). Few-shot means the model requires only a small number of examples to generalize and perform a task effectively. This helps reduce cost by eliminating the need for extensive fine-tuning or retraining, leveraging existing model capabilities to adapt to new tasks efficiently.

In the realm of chemistry, LLMs are increasingly utilized for a variety of tasks, including data processing, engineering, inference, and augmentation, in conjunction with various computational tools (Guo et al., Reference Guo, Nan, Liang, Guo, Chawla, Wiest and Zhang2023; Jablonka et al., Reference Jablonka, Schwaller, Ortega-Guerrero and Smit2024; M. Bran A et al., Reference Bran, Cox, Schilter, Baldassari, White and Schwaller2024). Despite these advancements, concerns about the explainability of these technologies continue to persist (Gallegos et al., Reference Gallegos, Vassilev-Galindo, Poltavsky, Martín Pendás and Tkatchenko2024), prompting further research into integrating LLMs with semantic technologies. QA systems have historically leveraged external knowledge bases, particularly through KG-based QA systems. These are designed to retrieve and reason over structured data from KGs to deliver precise and fact-based answers (Zhang et al., Reference Zhang, Dai, Kozareva, Smola and Song2018; Kim et al., Reference Kim, Kwon, Jo, Choi, Bouamor, Pino and Bali2023).

The emergence of retrieval-augmented generation (RAG) systems has taken this a step further by combining the reasoning capabilities of LLMs with the retrieval of up-to-date information from external sources (Lewis et al., Reference Lewis, Perez, Piktus, Petroni, Karpukhin, Goyal, Küttler, Lewis, Yih and Rocktächel2020). This allows RAG systems to generate more contextually relevant and accurate responses (Zheng et al., Reference Zheng, Zhang, Nguyen, Rampal, Alawadhi, Rong, Head-Gordon, Borgs, Chayes and Yaghi2023; Kang and Kim, Reference Kang and Kim2024). Using KGs as the foundation for information retrieval, recent studies have shown the great promise of KG-RAG to reliably handle knowledge-intensive and cognitive tasks (Sanmartin, Reference Sanmartin2024).

Another challenge for knowledge-intensive QA systems is handling of private, niche, or proprietary data, which is encountered in both industrial contexts and academic research. This necessitates a flexible QA system capable of integrating various data sources and domains while also allowing for the dynamic inclusion of new information. LLMs’ strong generalization ability and versatility are key to addressing these dual goals. For example, ChemCrow (M. Bran A et al., Reference Bran, Cox, Schilter, Baldassari, White and Schwaller2024) is a tool-calling agent capable of incorporating information pulled from a mixture of public and private data sources and computational tools, including the PubChem database and the RoboRXN platform by IBM Research (IBM, 2021). It does so by employing an LLM pretrained for the tool-calling task to orchestrate when to use which external tool and how to process and combine the results to form a coherent response (Schick et al., Reference Schick, Dwivedi-Yu, Dessi, Raileanu, Lomeli, Hambro, Zettlemoyer, Cancedda, Scialom, Oh, Naumann, Globerson, Saenko, Hardt and Levine2023).

Similarly, Marie is capable of querying across various domains and accessing information from distributed data sources within the fields of combustion kinetics and crystalline zeolitic materials (Pascazio et al., Reference Pascazio, Tran, Rihm, Bai, Mosbach, Akroyd and Kraft2024; Kondinski et al., Reference Kondinski, Rutkevych, Pascazio, Tran, Farazi, Ganguly and Kraft2024b). However, the previous version of Marie relies on supervised fine-tuning for its semantic parser, which necessitates retraining whenever it needs to integrate with a new knowledge domain in TWA. In contrast, the in-context learning capability of LLMs (Brown et al., Reference Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal, Neelakantan, Shyam, Sastry, Askell, Agarwal, Herbert-Voss, Krueger, Henighan, Child, Ramesh, Ziegler, Wu, Winter, Hesse, Chen, Sigler, Litwin, Gray, Chess, Clark, Berner, McCandlish, Radford, Sutskever and Amodei2020) offers a promising approach to expanding Marie’s coverage across TWA’s domains without the need for retraining. This capability allows LLMs to perform tasks based solely on task demonstrations provided at test time, without updating model weights—particularly, if coupled with advanced entity linking algorithms (Nie et al., Reference Nie, Zhang, Wang and Liu2024).

3.1. Updates to OntoMOPs

To include MOP knowledge in our chemistry QA system Marie and extend it for better user interactivity, the MOP knowledge base needed to be restructured and extended first. As a first step, we made adjustments to the original OntoMOPs ontology to improve robustness and ease of querying. The changes concern two main aspects: the storage of geometry data and the elimination of potential data redundancy. In a second step, the MOP KG was enriched with 370 new geometries of machine-predicted MOPs in addition to the 151 existing geometries of previously synthesized MOPs. These molecular geometries were deduced from information represented in the KG and will help researchers to better visualize these structures and screen possible synthesis candidates.

The updated ontology is shown in Figure 2. Its core concepts form a rectangle: MOPs can be classified by their geometric assembly models made up of distinct GBUs as which a variety of CBUs can function (Kondinski et al., Reference Kondinski, Menon, Nurkowski, Farazi, Mosbach, Akroyd and Kraft2022). These four core concepts now provide access to a range of geometric and molecular properties alike.

Figure 2.

Illustration of the terminological component (TBox) of the MOP chemistry domain in TWA and its related ontologies. Core concepts are shown in bold.

In the original implementation, geometry data of MOPs and CBUs are provided as XYZ documents or XYZ-formatted strings in the KG. A potential limitation of this method is that a string can, in principle, exceed the stringent length limits imposed by the KG engine—for example, in the case of a very large chemical superstructure. An alternative implementation entails the instantiation of every atom in an MOP or CBU structure and linking these atoms to an intermediate Geometry node, which is then connected to an MOP or CBU instance via the hasGeometry predicate, as done in the OntoSpecies domain of TWA (Pascazio et al., Reference Pascazio, Rihm, Naseri, Mosbach, Akroyd and Kraft2023). However, doing so for large MOP structures could introduce an overwhelming number of triples and, consequently, may slow down KG operations. In this work, we make a compromise between limiting the number of instantiated triples and avoiding storing long strings directly in the KG by moving the storage of the geometry data to XYZ files on disk. These files are hosted on a web server so that they are accessible on the Internet via URLs, which are discoverable in TWA through hasGeometryFile links to Geometry nodes.

The original assertion component created redundancy in the assignment of IRIs for instances of assembly models and GBUs, necessitating the postprocessing of aggregate queries. In the new implementation, redundant entries were merged or removed, allowing for simpler traversal of the KG without lengthy queries. On a terminological level, our effort to increase interoperability and overlap between chemical TWA ontologies, particularly with the renewed implementation of OntoSpecies (Pascazio et al., Reference Pascazio, Rihm, Naseri, Mosbach, Akroyd and Kraft2023), has facilitated the reuse of general-purpose concepts. As illustrated in Figure 2, this reuse covers many concepts related to shared molecular properties and literature provenance. This way, we intensify the interlinkedness of TWA and simplify agentic data curation and the training of Marie modules.

3.2. The architecture of Marie TWA

A QA system for TWA is not only required to map user intents to a machine-readable format accurately but it must also identify the correct data repository that contains the requested information. The latter stipulation arises from TWA’s compartmentalization of its data into distinct triplestores to allow domain experts to own and manage them independently. Earlier versions of Marie struggle with the dynamic nature of TWA, as its semantic parser relied on supervised fine-tuning (Tran et al., Reference Tran, Pascazio, Akroyd, Mosbach and Kraft2024), requiring resource-intensive retraining. This limitation not only impedes Marie’s scalability but also poses the risk of catastrophic forgetting (Hadsell et al., Reference Hadsell, Rao, Rusu and Pascanu2020). In contrast, the current version of Marie is designed with a more agile and adaptable architecture, ensuring continued support for existing chemical domains within TWA while seamlessly extending coverage to new domains, such as OntoMOPs.

To achieve this, we set up a KG-RAG system based on a modular architecture and adapted in-context few-shot learning methods to it. As depicted in Figure 3, Marie’s online workflow comprises three main components:

1. 1. The input rewriter aligns all physical quantities mentioned in the input question to the unit systems in our knowledge base.

2. 2. The semantic parser jointly generates the logical form of a SPARQL query, detects the surface forms of entities present in the input question, and determines the triplestore to execute the query.

3. 3. The response generator presents the structured SPARQL response and LLM-generated styled text, accompanied by visualization of the 3D structures of any invoked chemical entities.

Figure 3.

Architecture of “Marie,” comprising one offline indexing stage and three online stages, namely input rewriting, semantic parsing, and response generation.

Both the quantity recognizer and semantic parser are powered by LLMs prompted with in-context examples; the exact structure of these prompts is available in the Supplementary Materials. While the LLM prompt for the physical quantity recognizer is fixed, the semantic parser dynamically adapts to the input question by incorporating only the $ {k}_{\mathrm{demonstrations}} $ most relevant semantic parsing demonstrations and $ {k}_{\mathrm{KG}\_\mathrm{relations}} $ most relevant KG relations. This approach is key to Marie’s rapid integration with new knowledge domains because only a small number of semantic parsing demonstrations and KG relations need to be prepared, unlike the relatively larger training dataset required for supervised fine-tuning. Additionally, the on-demand retrieval of the most relevant elements for prompt construction ensures that the prompt is as compact as possible to fit within the context window of common LLMs while also saving processing time. Relevance is measured by the cosine similarity of their Sentence-BERT embeddings (Reimers and Gurevych, Reference Reimers, Gurevych, Inui, Jiang, Ng and Wan2019) using the all-mpnet-base-v2 variant. We use OpenAI’s gpt-4o-mini-2024-07-18 model for in-context learning and the Redis Community Edition for all retrieval needs. These techniques allow us to construct Marie as a modular KG-RAG system that leverages pretrained LLMs without requiring fine-tuning or retraining itself. This not only drives down the cost of operating Marie as well as adding more subjects but also ensures response accuracy.

Marie’s entity linking component uses different strategies depending on the entity class:

• • Inverted index lookup for entities with well-defined labels, for example, chemical species with their IUPAC names, molecular formulas, and SMILES strings.

• • Semantic search for entities that represent concepts or categories, for example, chemical classifications.

• • RDF subgraph matching for more complex entities that are conceptually defined by their relationships with other entities, for example, assembly models composed of GBUs (Kondinski et al., Reference Kondinski, Menon, Nurkowski, Farazi, Mosbach, Akroyd and Kraft2022).

Compared to earlier versions, this multistrategy approach has been refined to accommodate the diverse and growing range of entities within TWA, particularly the complex entities in the MOP chemistry domain. In the Supplementary Materials, we provide a summary of entity linking strategies and an illustration of RDF subgraph matching.

The response generation component in Marie has been enhanced to provide more comprehensive and user-friendly outputs. Marie’s structured output is presented in both JSON and tabular format, allowing users to view the raw SPARQL response in JSON and the formatted version in a table. On top of this, the natural language text generated by an LLM explains the results in a more accessible manner. A major update in the current version is the visualization of intricate chemical structures like MOPs; this is done using the library 3Dmol.js (Rego and Koes, Reference Rego and Koes2014). This feature not only broadens the utility of the QA system by making complex chemical data more tangible but also enhances the overall user experience, allowing researchers to engage with the data more interactively.