Plain language summary
In this article, we explore how far automated methods can support the early stages of narratological analysis for Old Church Slavonic (OCS) texts, starting with a core prerequisite: identifying characters. Being able to reliably detect personal names is essential for later narratological questions, such as who acts, who speaks to whom, what roles characters play in the plot, and what patterns of interaction emerge. We operationalize character identification by conducting named entity recognition (NER), a natural language processing (NLP) task that tags entities like persons, places, and organizations. We focus on the first category, and, especially on proper names. We address this by working with two versions of Lives of Saints, a 10th-century version of hagiographic and homiletic texts translated from Greek found in the Codex Suprasliensis, and a 16th-century version found in the East Slavic Great Menaion Reader. OCS is a useful test case for automated text analysis because it is a low-resource language with limited digitized data and few off-the-shelf tools. It also requires ad hoc preprocessing because of older Cyrillic characters, superscripts, and a wide range of diacritics, where tokenization and normalization choices can have a big impact. We conduct some experiments with the ground truth data, highlighting differences between different Lives regarding, among others, the use of direct speech. To identify person names automatically, we first conduct experiments with NLP methods. We test a “transfer” approach: we run an English NER model on an English translation of the OCS source text and then try to align the detected names using word embeddings. This performs poorly because the embeddings have limited coverage, especially for proper names, and because differences between text versions and normalization choices lead to mismatches. As a fallback, we consider part-of-speech tagging to flag proper nouns, but this is only a partial solution and underlines the broader scarcity of reliable NLP resources for OCS. We include large language models (LLMs) to overcome these shortcomings. First, we prompt GPT-5 (Thinking mode) under zero-shot condition and ask the model to individuate themes in the given input texts (topic modeling), which we evaluate qualitatively with experts. Second, we develop a retrieved augmented generation coding pipeline with GPT-4o. In plain terms, the model is given smaller chunks of the input texts and is asked to find names so we can extract them and group together different spellings of the same person. We then index the text so we can retrieve the passages that mention a given character. Finally, we use these retrieved passages to draft short character profiles. Our experiments suggest that LLMs can meaningfully assist early stages of narratological analysis of OCS despite orthographic and morphological variation, especially when used in a controlled workflow.
However, the approach is currently limited by a focus on named entities, potential contamination in the training data, and the need to control verbosity and formatting, so expert review remains essential. Future work may want to add external linguistic validation, extend the pipeline to events and direct speech, and evaluate open LLMs to enhance reproducibility.
Introduction
In the last years, Computational Literary Studies (CLS) in general and computational narratology in particular have rapidly evolved. New models for different languages and purposes have been created, connections to the natural language processing (NLP) community have been strengthened, large datasets compiled, and sophisticated methods developed. In CLS, the advent of large language models (LLMs) both as an analytical tool and as a methodological challenge has been met proactively and with great interest and success. However, a good portion of the research is concerned with narrative texts written in modern western languages, most notably English. Despite the recent advances in multilingual and historical NLP, we still witness a resourcedness gap between modern western languages and other languages. This holds especially in situations where the texts under investigation are written in varieties that are pre-modern and non-western at the same time. In the current article intended as a pilot study, we explore ways for computational narrative analysis of such pre-modern and non-western sources, particularly Old Church Slavonic (OCS)Footnote 1 Lives of Saints, with a specific focus on identifying and analyzing persons, their interactions, and narrative roles. This is particularly challenging because Church Slavonic (CS) has complex morphology and high orthographic variation, and manuscripts often contain numerous diacritics. Crucially, punctuation is sparse or inconsistently used, so sentence boundaries are hard to identify, which complicates downstream tasks for computational narrative analysis, especially tracking persons. In order to tackle these challenges, we apply already existing resources, specialized NLP tools, and LLMs. Both the NLP- and the LLM-based approach face specific challenges when it comes to working with pre-modern, non-western texts: Models and training data for specific NLP tasks are scarce or not available at all, some available models struggle when confronted with the inherent variation of pre-modern texts; smaller LLMs are not really multilingual, and even multilingual Frontier LLMs struggle with historical low-resource languages and are heavily biased toward western, modern standard languages. In our article, we explore the limits of traditional NLP and we experiment with LLMs. While LLMs remain limited in generating text in low-resource languages, such as OCS and related varieties and still confuse different pre-modern Slavic varieties (Lendvai et al. Reference Lendvai, Reichel, Jouravel, Rabus and Renje2025), their partly language-agnostic representational spaces and their strong semantic capabilities (Jannidis et al. Reference Jannidis, Kleymann, Schröter and Zinsmeister2025) suggest a significant potential for narrative analysis of low-resource pre-modern non-western textual data.
The article is structured as follows: First, we discuss related work and give basic information about our main source, the OCS Codex Suprasliensis; the next section is devoted to the discussion of the data used and the methods applied. Subsequently, we present and discuss our experiments, followed by our conclusion.
Related work
The last years have witnessed a steep increase in computational approaches to narratology, both from the viewpoint of CLS and Computer Science. Notably, shared tasks for creating narrative annotation schemas have been elaborated and commented on https://sharedtasksinthedh.github.io/. Recently, deep learning and AI methods have also been applied to the study of narratives (Piper and Bagga Reference Piper and Bagga2022). The advent of LLMs has also been reflected in the latest years, where experiments with LLMs for narrative analysis are conducted (Campos et al. Reference Campos, Jorge, Jatowt, Bhatia and Litvak2024; Jenner et al. Reference Jenner, Raidos, Anderson, Fleetwood, Ainsworth, Fox, Kreppner and Barker2025; Piper and Bagga Reference Piper, Bagga, Lal, Clark, Iyyer, Chaturvedi, Brei, Brahman and Chandu2024). A prominent example is the fundamentally revised second edition of the monograph Mani (Reference Mani2026). Here, Mani systematically explores the capabilities of frontier LLMs (at the time of writing GPT-4o or Claude Sonnet 3.5) for different tasks such as annotation according to the NarrativeML schema previously developed by him, or detecting plausible and implausible violations of narrative structure. He quotes several studies devoted to different languages, but predominantly uses modern English data for his experiments. The contributions mentioned here primarily focus on LLM-driven narrative analysis of text written in modern languages. Pre-modern, non-Western narratives are rarely in the focus of such studies. To our knowledge, there are no studies devoted to quantitative narrative analysis of pre-modern Slavic texts.
The Codex Suprasliensis
The OCS Codex Suprasliensis (Figure 1), written in the 10th century, is the largest extant OCS textual source, a “Champion” of OCS manuscripts (Neumann Reference Neumann2025). The manuscript, which contains hagiographic and homiletic text translated from Greek, has been included in the UNESCO’s Memory of the World listing in 2007. It has caught the attention of generations of scholars, predominantly from a philological, textological, and linguistic viewpoint. There are also interesting qualitative studies that touch upon issues related to narratology (e.g., Dekker Reference Dekker2021).
Beginning of the Life of Basiliscus in the Codex Suprasliensis.
Figure 1 Long description
At the top, a block of Old Church Slavonic text is written in dark ink, arranged in evenly spaced horizontal lines. Midway down, a horizontal decorative band with interlaced geometric motifs separates the upper and lower text blocks. Below the band, the text resumes in the same script, beginning with a large, ornate initial letter B on the left margin, filled with intricate linework. The lower text block continues in straight lines to the bottom margin. The parchment shows signs of age, with a torn upper right corner and faded edges. The folio number 28 is written in red in the upper right.
Recently, Neumann (Reference Neumann2025) applied an array of NLP methods to the Codex Suprasliensis for the first time, thus uncovering interesting linguistic and structural details. Moreover, the text is part of treebanks, such as TOROT (Eckhoff and Berdicevskis Reference Eckhoff and Berdicevskis2015) and PROIEL (Haug and Jøhndal Reference Haug, Jøhndal, Sporleder and Ribarov2008), as well as Universal Dependencies (Nivre et al. Reference Nivre, de Marneffe, Ginter, Hajič, Manning, Pyysalo, Schuster, Tyers and Zeman2020). As opposed to the majority of the extant OCS manuscripts, the Codex Suprasliensis does not contain translations of biblical books, but rather – as mentioned before – homiletic texts and Lives of Saints, making it also relevant for different kinds of narrative analysis.
While it is obvious that the Codex Suprasliensis is a prime candidate for pre-modern Slavic computational narrative analysis using both established NLP approaches and LLMs, there is an important caveat that needs to be taken into account: Since different digital versions of the Codex Suprasliensis have been available online for many years, one needs to assume that they have been scraped by most LLM providers for pretraining, leading to potential data contamination. This is why we resolved to complement our data sources with CS textual data not publicly available on the Internet. This new data essentially contains the same texts, but is written in a younger CS variety with, above all, different orthography (see below).
Data and methods
Files and preprocessing
The data at our disposal are rather messy, which is typical for real-world Digital Humanities (DH) projects. First, we use the fully manually tagged .conllu-file of the Codex Suprasliensis available at https://github.com/syntacticus/syntacticus-treebank-data. It serves as our ground truth (GT) and can be used for applying various NLP and corpus linguistics tasks. While it is convenient to have GT data available for evaluation, it can also be problematic, because the GT has been used as training data for both specialized NLP models and – most probably – for the training of LLMs as well. This is the reason why, for some of the experiments, we utilized a later CS version, namely, from the 16th-century East Slavic Great Menaion Reader (Velikie minei čet’i (VMČ), see Weiher, Šmidt, and Škurko Reference Weiher, Šmidt, Škurko, Daiber, Daiber, Dianova, Keller, Kobjak, Kostjuchina, Minčeva, Pliguzov, Serebrjakova, Šul’gina, Voss and Weiher1997, Figure 2).
Beginning of the Life of Basiliscus in the Great Menaion Reader (VMČ).
Figure 2 Long description
The manuscript photo displays an open spread with two aged pages. Each page is divided into two vertical columns, totaling four columns of text. The writing is in black ink, with occasional red ink used for headings or initials, especially at the start of paragraphs or sections. The script is Cyrillic, written in a consistent, compact hand. Marginal notes and corrections appear in smaller script along the outer and inner margins of both pages. The parchment shows signs of wear, including stains and discoloration, particularly along the edges and the central fold. No illustrations or decorative borders are present; the focus is entirely on the textual content.
Table 1 shows an example of the Slavic text we are dealing with (from the Codex Suprasliensis and the VMČ version of the Life of Basiliscus).
Comparison of different pre-modern Slavic versions of the Life of Basiliscus
Table 1 Long description
Starting from the top row, the left column contains Old Church Slavic text from Codex Suprasliensis, the middle column presents Great Menaion Reader variants, and the right column provides English translations. The first row reads ‘And I have written your name first among the martyrs.’ The second row translates as ‘who are with you; you have.’ The third row states ‘become sad that you were called.’ The fourth row reads ‘last. But you shall surpass many.’ The fifth row says ‘Go and say goodbye to your relatives.’ The sixth row is ‘And having come, you will receive martyrdom in Co-.’ The seventh row continues ‘mana. But do not fear tortures, for I.’ The eighth row concludes ‘am with you.’ The ninth row is blank in Codex Suprasliensis and English, but Great Menaion Reader contains ‘бoю.’ Each row aligns the corresponding phrases across the three versions, highlighting textual and linguistic differences.
As can be seen, there are numerous ancient Cyrillic letters, superscripts, and combining as well as non-combining diacritics that, despite being part of different standard Unicode codepages, make it hard for different NLP tools, packages, and preprocessing pipelines to be applied efficiently. The same holds for language models, where tokenization might lead to problems.
Next, we used the Greek (reconstructed) original text. It features diacrictics typical for polytonic Greek as well. However, the degree of orthographic variation is considerably smaller. Moreover, comparably more NLP resources for ancient Greek exist than for pre-modern Slavic.
Our third and final textual variant consists of an English translation originally designed for purposes related to the creation of the Codex Suprasliensis online edition.Footnote 2 The files kindly provided to us by the creators of the online edition are legacy Word files with numerous color codings, footnotes, and comments, making alignment processes challenging. Nevertheless, since such a situation – some computational resources may or may not exist, but the bulk of available data has originally been prepared for other purposes than quantitative analysis (such as book publications) and is fuzzy/messy – is rather common for real-world DH projects, we deliberately use this messy data in order to show what can be achieved using computational methods and legacy data.
Corpus variants and contamination control
We analyze pairs of uncontaminated (not publicly posted online) and publicly known transcriptions of selected hagiographies, each in two orthographic settings: (i) non-normalized (retaining diacritics and scribal conventions) and (ii) lightly normalized (reducing select marks and harmonizing frequent variants). This design makes it possible to separate the effects of orthography from any potential pretraining exposure. It also foregrounds how strongly small orthographic differences may affect recognition in languages that remain low-resource and largely outside mainstream NLP pipelines.
NLP-based experiments
Analysis using ground truth data
As the Codex Suprasliensis is part of the OCS Universal Dependencies dataset, an annotated GT version of the text can be used for narratological analysis. Lemmas, part-of-speech (POS) and full-morphology tags, and dependency parsing can serve as established descriptive methods to shed light on the narrative organization of different hagiographic texts. Figure 3 shows the most frequent content words of the Life of Basiliscus. As can be seen, the most frequent content word is рещи “to say,” indicating that direct speech plays a crucial role for the narrative organization of this particular vita. Analysis of the inflected verb forms revealed that the most frequent type is the third person singular form рече “he said,” indicating that the martyr and his opponents are the ones speaking. In other Lives, for example, the Life of Isaacius, the relative frequency of the verbum dicendi рещи is considerably lower, hinting on differences in the narrative structure of the individual Lives.
Most frequent content words in the Life of Basiliscus.
Figure 3 Long description
The bar chart displays word tokens along the x-axis, each labeled in Cyrillic script, and frequency on the y-axis ranging from zero to thirty-five. From left to right, the tallest bar is labeled 'рещи’ with a frequency of thirty, followed by 'богъ’, 'василискъ’, and 'воѥвода’, each with frequencies between twenty and twenty-five. The next group includes 'свѧтъ’, 'жрьти’, 'господь’, 'глаголати’, and 'видѣти’, with frequencies between thirteen and seventeen. The remaining bars, including 'ити’, 'сътворити’, 'жрьтва’, 'воинъ’, 'храмъ’, 'повелѣти’, 'чюдо’, 'огнь’, 'молити’, and 'молитва’, have frequencies ranging from five to ten. All bars are solid blue, and the chart background is light gray with gridlines.
In order to provide insight into the relations between the persons in the vitae and their actions, and to compare the narrative structures of selected Lives of Saints present in our source, we used R’s quanteda (Benoit et al. Reference Benoit, Watanabe, Wang, Nulty, Obeng, Müller and Ma2018) to create network plots. We used the lemmas and filtered out stopwords, such as prepositions and conjunctions.
As can be seen in Figure 4 depicting a textplot network of the OCS version of the Lives of Paulus and Juliana, богъ “God” is in the center of the network.Footnote 3 He is strongly connected not just with the two martyrs (Павьлъ “Paulus” and Иоулиани “Juliana,” respectively), but also with the cruel tormenter Aurelianus (Аѵрилиꙗнъ). As for reported vs. direct speech, it can be seen here that the two verbs with the meaning “to speak,” глаголати and рещи, do not have a strong link to the martyrs, suggesting direct speech is not particularly dominant in the martyrs’ narrative. However, in other Lives such as the Life of Basiliscus, the connections between the verba dicendi and the martyr Basiliscus (Василискъ) are much stronger, indicating a greater importance of direct speech in this particular legend (Figure 5).
Textplot network with lemmatized ground truth of Paulus and Juliana.
Figure 4 Long description
At the center are two closely positioned nodes labeled with Cyrillic script: Богъ and рещи. From these, numerous curved lines extend outward, connecting to peripheral nodes labeled мъногъ, христосъ, повелѣти, павьлъ, огнь, аѵрилиꙗнъ, глаголати, иоулиꙗни, свѧтъ, отьць, сътворити, жена, and видѣти. The central nodes have the highest number of connections, with some outer nodes like павьлъ, огнь, and аѵрилиꙗнъ also showing multiple links. The network structure is radial, with the majority of connections originating from the core and fewer interconnections among the outer nodes.
Textplot network with lemmatized ground truth of Basiliscus.
Figure 5 Long description
At the center is a node labeled with Cyrillic text resembling 'рещи.’ From this core, edges extend outward to ten peripheral nodes, each labeled in Cyrillic. The left side features three nodes with single direct links to the center: 'сътворити’ at the upper left, 'прити’ at the middle left, and 'ити’ at the lower left. The right side is densely interconnected, with nodes labeled 'господь’ at the upper right, 'глаголати’ to its lower left, 'жрьти’ to the right, 'василискъ’ below center, 'воѥвода’ to the lower right, 'богъ’ below 'василискъ,’ 'видѣти’ at the bottom, and 'свѧтъ’ at the lower right. Many right-side nodes are linked not only to the center but also to each other, especially around 'василискъ,’ which connects to 'господь,’ 'глаголати,’ 'жрьти,’ 'воѥвода,’ 'богъ,’ and 'свѧтъ.’ The network visually emphasizes the centrality of the core node and the clustering of relationships on the right.
Another well-known legend, the Life of Isaacius, shows yet another picture (Figure 6).
Textplot network with lemmatized ground truth of Isaacius.
Figure 6 Long description
At the center are two main nodes labeled in Cyrillic: 'цѣсарь’ and 'свѧтъ’. Lines radiate outward from these central nodes to connect with peripheral nodes, each labeled in Cyrillic script. The peripheral nodes include 'исаакии’, 'господь’, 'речи’, 'глаголати’, 'мъногъ’, 'отьць’, 'градъ’, 'црькꙑ’, 'вѣра’, and 'богъ’. Each line represents a relationship, with some peripheral nodes connected to both central nodes and others connected to only one. The network structure highlights the centrality of 'цѣсарь’ and 'свѧтъ’, with the densest connections converging at these points. The curved lines indicate the direction and strength of association between terms, forming a web-like structure with the highest density at the core and sparser connections at the edges.
As can be seen here, the verba dicendi глаголати and рещи are not connected at all with our saint Isaacius (Исакии) and neither with words used to name him such as отьць “father,” or epitheta such as свѧтъ “holy.” The word token most connected with verba dicendi is цѣсарь “emperor.” N-gram analysis and qualitative inspection of the co-occurrences of цѣсарь showed that there are several different emperors mentioned, both pious Christians and evil adversaries of holy men such as Isaacius. Using quanteda’s keyness analysis feature, a score to quantify the differences of a target corpus to a reference corpus (see also Bondi and Scott Reference Bondi and Scott2010), the most significant differences on word token level between the two texts can be uncovered (Figure 7).
Keyness analysis of Basiliscus vs. Isaacius.
Figure 7 Long description
The chart displays twenty-four horizontal bars aligned along a central vertical axis labeled G2. Bars extending right are dark blue and represent terms more associated with Basiliscus, with the top two bars labeled with Cyrillic script for ‘Basiliscus’ and ‘voivode’ showing the highest positive G2 values, followed by terms such as ‘sacrifice’, ‘say’, ‘temple’, ‘warrior’, and others. Bars extending left are gray and represent terms more associated with Isaacius, with the longest bar labeled ‘tsar’, followed by ‘church’, ‘faith’, ‘Isaacius’, and others. The bars decrease in length as they move away from the center, indicating lower keyness. All labels are in Cyrillic script and are positioned at the end of each bar. The chart visually contrasts the lexical prominence of terms between the two subjects.
Unsurprisingly, the main antagonists, Basiliscus and the voivode (Василискъ, воѥвода), are the tokens most significantly different from the Isaacius reference corpus, as can be seen in the blue area of the plot. The mentioning of the emperor (цѣсарь) stands out in the gray Isaacius reference corpus. Verba dicendi are only relevant in the Basiliscus corpus (рещи), supporting the results about different narrative organization by means of direct versus indirect speech obtained by the network analysis above. Moreover, keyness analysis can uncover differences in both central conceptual features and cohesive devices. For instance, one of the main conflicts leading to the martyrdom of Basiliscus was the issue of him being forced to sacrifice (жрьти) to pagan gods and his refusal to do so. This specific topic can be uncovered using keyness analysis. Conversely, the Life of Isaacius prominently features the cohesion device тъгда “then,” indicating that differences in the composition and the syntactic architecture of different Lives can be uncovered using this method.
Analysis using established NLP methods
In NLP, the task of identifying and appropriately tagging linguistic units that refer to persons is known as named entity recognition (NER). Since NER is a common NLP task, a natural first approach is to apply an off-the-shelf model that has been trained on a large corpus. For English, widely used frameworks, such as spaCy (Honnibal and Montani Reference Honnibal and Montani2017), AllenNLP (Gardner et al. Reference Gardner, Grus, Neumann, Tafjord, Dasigi, Liu, Peters, Schmitz, Zettlemoyer, Park, Hagiwara, Milajevs and Tan2018), and NLTK (Bird and Loper Reference Bird and Loper2004), provide robust pre-trained taggers that can accurately identify entities, such as persons, organizations, and locations. In low-resource languages, however, this straightforward approach is rarely feasible. Such languages often lack comprehensive annotated corpora, pre-trained models, and standardized pipelines for preprocessing, tokenization, and normalization. Inconsistent treatment of text, particularly in languages with diacritics, archaic orthography, or complex morphological structures like OCS, can lead to divergent outputs across NLP tools. These divergences reduce reproducibility and hinder interoperability between frameworks, making it difficult to achieve consistent results or to validate findings across studies. In historical languages, preprocessing therefore plays a central role in the NLP workflow; any variation in normalization or tokenization can dramatically affect downstream analyses, including NER.
Without an off-the-shelf NER tagger, projection from high-resource languages to OCS is a natural first strategy for NER. In our initial experiment, we applied spaCy’s pre-trained model to identify person entities in the English translation of the Life of Basiliscus. We then attempted to align these detected entity tokens with the corresponding OCS text using monolingual OCS word embeddings provided by Pedrazzini (Reference Pedrazzini2023). For the alignment of the English and OCS vector spaces, we tested the MUSE framework (Lample et al. Reference Lample, Ott, Conneau, Denoyer, Ranzato, Riloff, Chiang, Hockenmaier and Tsujii2018), a tool designed to perform unsupervised mapping of embeddings between languages.
Despite these efforts, only a limited portion of tokens in the three OCS text versions could find a direct match in the embeddings (see Figure 8), and proper nouns, the key elements for our narratological analysis, were especially underrepresented.
Ratio of tokens and types matched with occurrences in the OCS embeddings of Pedrazzini (Reference Pedrazzini2023) across the Life of Basiliscus in three versions: Codex Suprasliensis (ground truth), VMČ (normalized), and VMČ (non-normalized).
Figure 8 Long description
Top left panel shows ‘Tokens, not lemmatised’ with Suprasliensis, VMČ Norm, and VMČ Not Norm on the x-axis. Green bars (Found) are lowest for Suprasliensis, slightly higher for both VMČ versions, but all are under 30 percent. Red bars (Not Found) dominate. Top right panel, ‘Tokens, lemmatised’, shows green bars increasing: Suprasliensis has the highest proportion found, VMČ Norm and VMČ Not Norm are lower but still improved over the non-lemmatised case. Bottom left panel, ‘Types, not lemmatised’, shows almost entirely red bars for all three versions, indicating very few types found. Bottom right panel, ‘Types, lemmatised’, shows Suprasliensis with the highest green bar, VMČ Norm and VMČ Not Norm with lower but improved green bars compared to the non-lemmatised panel. All panels use green for Found and red for Not Found, with values on the y-axis from 0 to 100.
Several factors may help explain why the embedding-based alignment using Pedrazzini (Reference Pedrazzini2023) set of embeddings proved unsuitable for our goals. First, the embeddings predominantly capture the Codex Suprasliensis version, limiting their coverage of other textual variants. Second, they are lemma-based: while lemmatization improves token matching, it discards morphosyntactic information (e.g., case, gender, and number) that is crucial for our analysis. Third, proper nouns, the core focus of NER for persons, are generally absent from the embeddings’ vocabulary. Finally, lemmatization across slightly different historical stages and texts with similar content naturally introduces variation, which propagates and contributes to mismatched results. Taken together, these factors render embedding-based alignment less effective in this context, making the observed limitations an expected outcome rather than an anomaly.
An alternative approach is POS tagging, with particular attention to proper nouns. POS tagging is one of the few NLP tasks that has been explored for OCS, for example, with the CLSTM tagger (Scherrer, Mocken, and Rabus Reference Scherrer, Mocken and Rabus2018), which allows for basic morphosyntactic analysis of historical texts. It remains to be assessed how flexible the tagger is in handling new data formats or in reproducing expected formats. While POS tagging may allow for tentative identification of personal names, it is limited in scope and does not address other entity categories (e.g., locations or organizations). Overall, this highlights the current scarcity of NLP resources for OCS and the challenges inherent in applying standard tools to pre-modern, low-resource languages.
LLM-based experiments
Technical setup
The topic modeling experiments reported in this article were conducted using OpenAI’s gpt-5 model (release available as of September 2025) in the Thinking mode, accessed through the ChatGPT interface. The experiments were run with a zero-shot prompting setup. The model was used in its default configuration provided by OpenAI (temperature, max tokens, and other decoding parameters not explicitly adjusted).
The retrieval augmented generation (RAG)-based experiments reported in the following sections were conducted using gpt-4o model (OpenAI, release spring 2024), which has a knowledge cut-off of October 2023. Model calls were accessed exclusively through the OpenAI API, constrained to produce structured artifacts (JSON and CoNLL-U tables) rather than free-form text. The pipeline was implemented in Python 3.11 and makes use of a minimal stack of libraries: langchain (for orchestration and API wrappers), langchain_openai and langchain_community (for model and FAISS vector store integration), pandas (for tabular consolidation), and python-dotenv (for environment configuration). FAISS was employed for sentence-level and document-level retrieval indices, with OpenAI’s text-embedding-3-large providing the vector representations. All runs were executed in a reproducible environment with parameters recorded in JSON manifests. No model fine-tuning or additional pretraining was performed; the setup is deliberately lightweight and portable, designed to be adapted to comparable low-resource textual traditions.
LLM-emulated topic modeling
In an exploratory zero-shot setting, we used the prompt described by Piper and Wu (Reference Piper and Wu2025) with OpenAI’s gpt-5 model in Thinking mode, applied to parts of the Codex Suprasliensis text, namely, the Lives of Paul and Juliana. This yielded the results shown in Table 2. A qualitative inspection confirmed that the topics distilled by the LLM align with human expert judgments and that the prompt effectively reduced the risk of LLM verbosity. This exploratory study will be expanded by more systematic experiments in future work.
GPT-5-assisted topic modeling
Table 2 Long description
The header row contains Model and Topic output. The first row lists GPT-5 Thinking in the Model column and Christian hagiography and martyrdom in the Topic output column. The next row has an empty Model cell and the Topic output cell reads pagan persecution and anti-idolatry polemic with angelic miracles. The final row has an empty Model cell and the Topic output cell reads martyrdom of Paul and Juliana before Aurelian, followed by trials, torture, fire, healings, conversions in parentheses.
Retrieval augmented generation (RAG)
Architecture-guided LLM usage
In this series of experiments, LLMs are not used as end-to-end generators of narrative text, but as components within a staged workflow. Their role is to recognize patterns at several levels of abstraction – lexical, morpho-syntactic, and narratological – under conditions where traditional tools are limited. To make this effective for pre-modern, low-resource languages, the models are constrained to operate on bounded input segments and to return structured artifacts (such as JSON lists or token tables) rather than free prose.
These intermediate artifacts are subsequently consolidated by non-neural steps, while retrieval mechanisms supply appropriate local or broader context. Occasional corrective passes and optional fusions (e.g., epithet–name combinations) are integrated under conservative criteria. In this way, the generative frontier-models’ analytical capacity for flexible pattern recognition is channeled toward noisy input data with orthographic variation, transcriptional irregularities, or normalization differences, while keeping the resulting inferences tied to explicit textual witnesses in a low-resource language, which cannot be accessed with (naïve) direct prompting.
For example, in the Life of Basiliscus, surface forms, such as василискъ, василис҇ка, and василиском, are first extracted independently, then merged into a single figure cluster before narratological roles are inferred.
System setup and architecture overview of RAG-based experiments
The RAG-based experiments did not consist of prompting a model in the conversational manner of a chatbot, but of accessing LLMs through their APIs within a reproducible multi-stage workflow. Input texts were aligned with folio and line divisions and, where appropriate, lightly normalized. These segments were then submitted to a sequence of model passes that were constrained to produce structured artifacts, such as compact JSON key–value lists or CoNLL-U formatted annotations, rather than free-form prose.
Subsequent stages consolidated these artifacts by deterministic means: variant forms were clustered, occurrences tabulated, contexts indexed, and figure profiles assembled. The intention is to deploy the pattern recognition capacities of current models at different levels of abstraction while keeping interpretive drift under control. The orchestration is model-agnostic and can, in principle, be redirected to commercial or local systems with minimal modification.
This workflow is built to accommodate the idiosyncrasies of current models: repeated attempts are possible, chunk sizes can be modulated, and prompt formulations adjusted where needed. In this way, longer or more complex passages can be processed without breakdown, while outputs remain of a size suitable for inspection. Auxiliary mechanisms, such as edit-distance clustering or balancing heuristics, are employed where they appear to support stability or recall. In effect, the system distributes the tasks: LLMs provide flexible pattern recognition on low-resource language and morphologically complex data, while downstream consolidation aims to supply the consistency needed for comparative and narratological work.
LLM-assisted pipeline and content flow.
Figure 9 Long description
At the top, the process begins with ‘Ingest and Chunking’ in a blue box, where input text is split into chunks and language is detected, outputting ‘chunks’. An arrow leads down to ‘Keyword Extraction’ in pink, where the LLM finds names, titles, verbs, and transliterations, producing ‘candidates to hints’. The next step is ‘POS Tagging’ in pink, where the LLM generates CONLL-U sentences per chunk, outputting ‘raw CONLL-U’. From here, two arrows diverge: one leads to ‘POS Validator (optional)' in pink, where the LLM corrects obvious PROPN/FEATS, producing ‘validated CONLL-U’, and the other continues directly to ‘Person Extraction and Clustering’ in green. Both paths converge at this green box, where mentions are matched, normalized, and figures are clustered, outputting ‘sentences plus clusters’. The flow continues to ‘Build Search Indexes’ in yellow, where macro (chunk) and micro (sentence) search indexes are created, outputting ‘macro index plus micro index’. Next is ‘Narratology per Cluster’ in pink, where the LLM and both macro and micro RAGs analyze roles and relations, producing ‘profiles plus evidence’. The final step is ‘Epithet-Name Fusion’ in pink, where the LLM checks adjective plus PROPN as fixed epithets. At the bottom, a legend color-codes preprocessing (blue), LLM steps (pink), search indexing (yellow), and clustering slash heuristics (green).
LLM-assisted pipeline
For the results reported here, hand-crafted heuristics were disabled ; the pipeline was operated in an LLM-only mode and is visible in Figure 9. The stages are as follows:
-
P1. Segmentation and chunking: Folio/line-aware slicing of texts into compact units.
-
P2. Signal harvesting: A constrained LLM pass outputs compact JSON with potential person names, titles, and a small list of action verbs (plus tentative transliterations).
-
P3. POS/lemma tagging: A less constrained LLM pass yields token-level annotations in a UD CoNLL-U-like format (tokens, lemmas, and morphosyntactic features), which may then be corrected in a subsequent validator pass. Instabilities with long spans are handled by progressive tightening. An additional corrective pass may be invoked to adjust obvious errors in name tagging and inflectional features.
-
P4. Person extraction and clustering: Tokens are filtered for personal names (titles may be included or excluded). Lemma-level variants are softly merged using suffix reduction and string similarity, resulting in clusters with canonical forms.
-
P5. Search indexing: Two complementary indices are built: a macro index for longer chunks, and a micro index for sentences to provide the anchoring within precise textual witnesses from the input textual data.
-
P6. Narratological sketch: Concise JSON profiles (roles, relations, and acts) are generated per figure cluster, making use of both micro and macro retrieval.
-
P7. Epithet–name fusion (optional): Candidate adjective–name pairs are tested for stable epithet function and, if confirmed, folded back into the clusters.
Retrieval augmentation at two granularities
Retrieval was not employed in the straightforward manner of question-answering against a database, but rather as a contextual anchoring mechanism within a cycle of pre-processing and interpretive post-processing. The objective was not simply to extract “answers” from an external GT, but to position the model within carefully bounded contexts while still maintaining access to the larger textual field. In this way, narrative information could be drawn at different levels of abstraction, while at the same time exploiting the pattern recognition capacities of LLMs: on the one hand, retrieval of short local contexts supports precise anchoring of person mentions (e.g., identifying Basiliscus as the speaker in, e.g., prayer-focused passages), while, on the other hand, broader contextual retrieval helps situate these mentions within larger narrative segments (e.g., distinguishing interrogation scenes from miracle episodes in the Life of Basiliscus).
The combination allows the pipeline to mitigate fuzziness introduced by transcriptional variation while also capturing recurring narrative patterns, such as the alternation between trial scenes and miracle narratives typical of hagiographic discourse.
Two complementary indices were used. At a broader level, comparatively long spans were embedded, making it possible to identify thematic regions, such as interrogations or miracle accounts. This macro index provides orientation and cross-checking, though by itself would invite overly expansive generations.
At a finer level, smaller discourse units (sentences, sometimes with minimal surroundings) were indexed separately. This micro index is called upon when building narratological sketches: for each cluster of figures, a handful of short contexts are retrieved, keeping role assignments and speech acts close to explicit textual evidence and curbing over-interpretation.
Maintaining both views was considered important. The broader index helps ensure that no textual region is entirely overlooked, even when sources are noisy or only lightly normalized; the finer index anchors interpretive steps in more concise evidence. Several stabilizing measures – limiting retrieval windows, capping generations, and constraining outputs to predefined slots – were introduced to keep the procedure within useful bounds.
The design is meant to make layered use of the models’ pattern recognition capacities: at the lexical level (names and features), at the structural level (annotations and clustering), and at the narratological level (roles, relations, and functions). The approach illustrates one way in which low-resource-language texts may be processed without collapsing either into noise or uncontrolled elaboration.
Heuristic inputs: The system internally supports optional: (i) title handling modes (exclude/include/only), (ii) an additional validator pass that only upgrades obvious person tags, and (iii) simple rule seeds for common hagiographic names. These were kept off in the experiments to assess pure LLM behavior.
Non-uniform normalization improvements: Removing diacritic clutter helped merge superficial variants (Basiliscus), but also encouraged lemma drift across cases when contextual anchors were thin.
Reduction of hallucination by brevity: Constraining output length and schema yield terser, less speculative narrative sketches.
Outputs and their narratological usage
The outputs of the pipeline were written out as structured files at each stage.
These include tabular listings of extracted person mentions (with their surface variants, lemmata, and morphosyntactic features), consolidated cluster tables with canonical forms, and occurrence logs that record the folio and line positions of each mention. In addition, separate files collect the short textual snippets and broader passage contexts that were retrieved around these mentions. On this basis, narratively significant figures could be isolated, and compact narratological sketches generated that summarize roles, functions, and relationships in the hagiographic discourse. Together, these artifacts provide a reproducible set of intermediate and final data products that make it possible to trace not only which figures were detected, but also how they are attested in the individual textual witnesses.
The procedure is not limited to a single linguistic variety: it can be applied to different stages of CS and related Slavic traditions, as well as to Greek materials such as Byzantine Greek.
Against this backdrop, the modular architecture also points toward the possibility of scaling to the simultaneous incorporation of other low-resource languages (such as identical hagiographies in OCS and Byzantine Greek for comparative studies at scale) and to substantially larger corpora, so that a wider range of input documents can be subjected to similar narratological extraction and even comparative transcultural analyses, for example, for the occurrence of narrative strategies, such as stories within stories in culturally diverse corpora.
Normalization study (qualitative)
We compared the RAG-based pipeline outputs across orthography conditions. Here are some key observations:
-
• Name consolidation improved by normalization: Basiliscus was consolidated to a single cluster under normalization (correct), whereas two variants appeared in the unnormalized run; superscript letters had confused clustering previously. For instance, under normalization, surface variants, such as василискъ, василис҇ка, and василиском, were consolidated into a single cluster representing Basiliscus (see Table 3). In the unnormalized run, superscript letters led to two separate clusters for these forms, illustrating how orthographic noise can fragment person representations.
Table 3.Consolidated clusters with canonical lemmata, surface variants, and dominant morphosyntactic features
Table 3 Long description
The table has six columns: Cluster ID, Lemma, Surface variants, English, UPOS, and Top features. From the top row: clu:001, lemma мцс҇а, surface variant мцс҇а:1, English month, UPOS PROPN, features Case equals Gen, Gender equals Fem, Number equals Sing. clu:002, lemma василискъ, surface variants василискъ:2, василис҇ка:1, василиском:1, English Basiliscus, U P O S PROPN, features Case equals Nom, Gender equals Masc, Number equals Sing (2); Case equals Gen, Gender equals Masc, Number equals Sing (1); Case equals Ins, Gender equals Masc, Number equals Sing (1). clu:003, lemma максимиаоу, surface variant максимиаоу:1, English Maximian, U P O S PROPN, features Case equals Dat, Gender equals Masc, Number equals Sing. clu:004, lemma асклипиода, surface variant асклипиода:1, English Asclepiodotus, U P O S PROPN, features Case equals Acc, Gender equals Masc, Number equals Sing. clu:005, lemma агрипа, surface variant агрипа:1, English Agrippa, U P O S PROPN, features Case equals Nom, Gender equals Masc, Number equals Sing. clu:006, lemma госпожа, surface variant госпожа:1, English lady, U P O S PROPN, features Case equals Nom, Number equals Sing. clu:007, lemma христос, surface variants хс҇у:1, хвис҇:1, English Christ, U P O S PROPN, features Case equals Acc, Gender equals Masc, Number equals Sing (2). clu:008, lemma воевода, surface variant воевода:1, English voivode, U P O S PROPN, features Case equals Nom, Gender equals Masc, Number equals Sing. clu:009, lemma богъ, surface variant бгъ:2, English God, U P O S NOUN, features Case equals Nom, Gender equals Masc, Number equals Sing (2). clu:010, lemma мой, surface variant мои:1, English my, U P O S DET, features Case equals Acc, Gender equals Masc, Number equals Sing.
-
• Case-sensitive lemma drift: максимианоу was lemmatized to nominative in the normalized run although the surface case is dative; the unnormalized run handled this specific case better (see Table 4).
Table 4.Qualitative comparison across orthographic settings (expert judgment)
Table 4 Long description
Beginning at the top row, the table header lists Phenomenon, Unnormalized, and Normalized. The first row shows ‘Cluster месяца as person’ marked as present (F P) in both settings. The second row, ‘Basiliscus consolidated,’ is absent in Unnormalized and marked with a checkmark in Normalized. The third row, ‘CCase for максимианоу,’ is labeled better in Unnormalized and dative with an arrow pointing to lemma equals NOM in Normalized. The fourth row, ‘Voivode equals Agrippa coref,’ is split in Unnormalized and split with improvement noted in Test2 for Normalized. The fifth row, ‘Askleipiod type,’ shows LOC with an arrow pointing to PER in both columns. The final row, ‘Narratology verbosity,’ is high in Unnormalized and reduced after cap in Normalized. All arrows are described as pointing from the initial label to the target label within each cell.
-
• Over-interpretation: Maximian received a stronger profile than warranted by the text.
-
• Entity typing error: Askleipiod (a location) was treated as a person (see Table 4).
Exemplary results of RAG-based workflow
Tables 3, 5, and A2 illustrate the type of outputs generated by the staged LLM-based workflow. These results are particularly exciting in light of an often strangely misunderstood dichotomy between close and distant reading, a dichotomy that is frequently confused with broader debates about what AI does to intellectual endeavors and to scholarly research processes as such. The recent attention given to text-based LLMs has only heightened these conceptual tensions within text-based scholarly disciplines.
LLM-tagged entities as persons in three versions of the Life of Basiliscus: Codex Suprasliensis (ground truth), VMČ (normalized), and VMČ (non-normalized)
Table 5 Long description
Starting from the top row, the Codex Suprasliensis version lists gold P O S-annotated person names in Old Church Slavonic, including василиска, маѯимияноу, асʼклипиѡ̑да, агрипа, команѣхъ, аполонъ, серафимъ, амасиѧ, диоскоръ, маринъ, among others. The LLM POS-annotation for Codex Suprasliensis includes overlapping and additional names such as воѥвода, г҃и, г҃ь, аполоновъ, ст҃ꙑи, бъ҃. The second row for VMC has no gold annotation but LLM POS-annotation lists names like мцⷭ҇а, василиⷭ҇кⷶ, максимїаноу, асклипиѡда, въевода, агрипа, бг҃оу, ги҃, ісе҃, хе҃, бг҃ъ. The third row for VMC Normalized also lacks gold annotation; LLM POS-annotation includes мцс҇а, василис҇ка, максимианоу, асклипиода, агрипа, гс҇нѩ, госпожа, хс҇у, хвис҇, мои, бгъ. The table footnote clarifies that only Codex Suprasliensis has manual gold POS-annotation at present.
* At the present moment, we dispose only of manual gold POS-annotation of the Codex Suprasliensis.
The present experiments show, however, that a carefully curated and orchestrated architecture of LLM-based steps can move beyond this apparent dichotomy. The LLM-based workflow enables what may be described as “close reading, but at scale, and with AI”: exact narrative evidence is retrieved and anchored in its textual context, and this evidence can in turn feed into further qualitative research, not merely quantitative counting or clustering. Moreover, the same architecture can be scaled to substantially larger and diverse corpora extending the scope of narratological inquiry far beyond individual case studies.
This is both novel and methodologically significant. It requires an understanding of how to design and sequence LLM interactions within a workflow and technical knowledge of their inner workings and usability in the specific conditions of fuzzy, low-resource input data. At the same time, it demonstrates that LLMs, when embedded in an architecture attentive to philological detail, can extend rather than replace the traditional practices of scholarly interpretation. The result is a hybrid mode of analysis that remains grounded in textual witnesses, while enabling new kinds of systematic comparison across larger corpora.
By and large, qualitative analysis confirmed that the LLM recognized the names and functions of the main actors of the narrative more or less correctly. Above all, the main martyr Basiliscus (Василискъ, cluster 002) and his main antagonist, the voivode Agripa (Агрипа, cluster 006) are depicted with their most significant, roles, functions, behavior, and actions (see column notes in Table A2). The evidence snippets in Table A3 extracted from the CS source text generally support the assertions made by the LLM. However, the LLM failed to identify that the appellative voivode (воевода, cluster 008) is used to designate Agripa, leading to the creation of two separate clusters instead of one conflated cluster. Moreover, some non-persons have been erroneously identified as actors relevant to the narrative organization of the Legend, most prominently мцс҇а “of the month” (cluster 001) and мой “my” (cluster 010). It has also been incorrectly tagged as PROPN (Table 3), not as NOUN; the feature Gender=Fem is also incorrect. Even more strangely, the pronoun мой has been tagged as DET and, further down our pipeline, identified as one of the actors in the narrative. The ascribed role as supplicant and the function as a seeker of mercy fits with the context where Basiliscus prays and asks God to have mercy upon him (see Table A3 in cluster 010). However, the LLM did not establish the link that we are dealing here with Basiliscus’ direct speech during his prayer. Interestingly, both instances where the LLM ascribes active roles to non-persons have been classified as internal regarding their focalization (Table A2). Apparently, the direct speech present in some of the snippets identified by the LLM led to the erroneous assumption that the – in principle correctly identified – internal focalization needs to be connected to an actor previously unrepresented and not connected to external focalization. This indicates that direct speech plays a crucial role in how an LLM understands and analyzes the narrative organization of texts.
Other actors, such as богъ “God” (cluster 009), the almighty recipient of prayers, are correctly identified. However, it is not entirely clear whether or not the world knowledge of the LLM has played a role in this classification and/or the high certainty.
Overall, the LLM-based pipeline yielded both positive and negative results, suggesting that LLMs are best as an exploratory heuristic device, and a competent human in the loop is indispensable.
Conclusion, limitations, and outlook
The experiments demonstrate that current frontier LLMs can process non-normalized, non-western, and low-resource texts such as (Old) CS to a certain degree, enabling narratological analysis even under conditions of orthographic and morphological variability. Some results significant for narratological analysis could be obtained using established NLP and analysis methods – for example, keyness analysis as shown in Figure 7 suggests that direct speech is employed more extensively in the Life of Basiliscus as opposed to the Life of Isaacius, and the former Life seems to be more verb-heavy in general. However, overall, LLMs seem to offer a more malleable approach to dealing with this kind of data, allowing for flexible handling of varied orthographies, morphological richness, and incomplete or non-standardized annotations. Data contamination from publicly available sources cannot be ruled out, yet the results show that LLMs can be productively integrated into workflows for pre-modern textual corpora. At the same time, the scope of the present study has been limited to named entities in order to trace person occurrences in the Life of Basiliscus across different versions. This leaves aside narratologically relevant actors such as non-named entities. In addition, the outputs of LLMs often display considerable verbosity, which can complicate evaluation and formatting unless addressed through carefully refined prompt design.
Looking ahead, future work can be organized into three levels:
-
1. Measures aimed at improving the stability and reliability of outputs: This includes the integration of external linguistic cross-checking layers (e.g., grammars and textbooks) into the pipeline to flag rare morphological phenomena, stabilize edge cases, and bridge gaps between automatic extraction and expert philological review. Such refinements would also help mitigate difficulties arising from verbose or inconsistently formatted generations, which currently require repeated prompt adjustments to ensure usable output.
-
2. Extensions of the workflow that move beyond its initial nominal focus: Key directions here are modules for verbal categories (tense, aspect, and event sequencing) and for the detection of direct speech. These additions would broaden the range of narratological phenomena that can be systematically studied. Future iterations should also move beyond the present emphasis on named entities, so that implicit or indirect mentions of persons can be systematically incorporated into the analysis.
-
3. Research horizon: At the highest level, the architecture points toward a general strategy for handling pre-modern low-resource corpora: a scalable, reproducible orchestration of LLM steps designed to achieve “close reading, but at scale, and with AI.” Such an approach opens the way for comparative narratological analyses across larger and more diverse corpora, moving beyond individual case studies. Here, however, it remains difficult to disentangle whether model performance in some tasks reflects understanding of the input text or reliance on pre-acquired knowledge from pretraining. Future studies could explore this limitation.
The sections that follow exemplify these categories: technical refinements through external linguistic cross-checking, functional extensions to verbal and temporal phenomena, and narratological enrichment through the identification of direct speech.
Incorporation of further external linguistic cross-checking layers into the LLM workflow
A further methodological extension would be to introduce external cross-checking layers into the LLM workflow that make use of authoritative linguistic resources. In the case of OCS, this could include printed or (even badly transcribed) digital grammars or textbooks. Such material can be indexed or aligned in parallel with the primary corpus, and thereby serve as an additional reference layer against which model outputs are retrieved, generated, and compared.
The rationale for this design is that certain morphological or syntactic peculiarities are not easily captured by pattern recognition alone but are typical for low-resource language material and carry large narrative significance. They may occur only rarely, or appear in orthographic guises that resist standard-NLP clustering and normalization. By integrating a curated external layer, such edge cases could be flagged and cross-validated, bridging the gap between automatic extraction and expert philological review, without compute-costly fine-tuning or retraining of models. The intended outcome is not to automate philological judgment, but to stabilize the treatment of highly idiosyncratic or sparse, even yet singular phenomena and to document their occurrence in a way that remains traceable for further interpretive analysis.
Expanding the focus beyond nominal entities: Time, aspect, and event information
Because the starting point of our experiments was rooted in classical NLP tasks, the initial workflow inherited a strong focus on nominal data, in particular the extraction of persons, their narrative contexts, and the generation of structured artifacts such as CoNLL-U style annotations. This legacy emphasis has been useful for establishing baseline procedures, yet it represents only one dimension of narrative structure. Especially in Slavic but also in Greek sources, it is equally crucial to consider verbal phenomena: categories of tense or aspect, the encoding of temporal relations, and the sequencing and narrative perspectives of events. Extending the workflow toward these dimensions would allow the architecture to capture not only who is present in a text and how they are described, but also when and how actions unfold, and how temporal organization contributes to broader narrative strategies.
In practical terms, such an extension would be most naturally introduced between P3 (POS/lemma tagging) and P4 (person extraction and clustering), where verbal morphology and temporal markers are already accessible in tokenized form. Outputs from this stage could then flow into P6 (narratological sketch), where temporal information and event sequencing would enrich the role and relation profiles generated for individual figures.
Direct speech identification
An additional avenue of expansion for the LLM workflow concerns the detection of direct speech. In pre-modern Slavic sources such as OCS, quotation marks are generally absent, and the distinction between direct and indirect speech has to be inferred from lexical and syntactic cues. A frequent marker is the use of the finite verb form рече “s/he said” or the participle form глаголѧ “saying,” which often introduces speech by the martyr or an opponent.
Incorporating a stage for direct speech identification would enrich the pipeline beyond nominal and verbal information by situating utterances within the flow of events and by clarifying the distribution of voices in the narrative. From a technical standpoint, such an operation could be integrated after P3 (POS/lemma tagging), where verb forms and clausal structures are already available, and feed into P6 (narratological sketch), where the allocation of speech acts to individual figures forms a key part of the interpretive profile. The resulting annotations would allow closer analysis of dialogic structures and of the rhetorical balance between saints, adversaries, and divine agents in hagiographic discourse.
Deployment on HPC infrastructure and research sovereignty
For the present article, we limited the scope of our experiments to the exploration of closed-source LLMs of the OpenAI family. Future work should test open-weights models, such as GLM-4.7, Qwen-3, BLOOM, LLaMA, and DeepSeek in order to ensure reproducibility and comparability. While commercial APIs remain useful for prototyping, long-term narratological work gains from relocation to dedicated high-performance computing resources. Such environments provide not only the scale and stability required for larger corpora processing at manageable cost, but also data and compute sovereignty: scholars retain control over parameters, storage, and execution rather than delegating them to potentially opaque market services. This shift is particularly significant for the humanities, where technical infrastructures increasingly shape the very conditions of research. By anchoring experiments in institutionally governed HPC, one establishes a reproducible and academically accountable foundation on which large-scale language model studies such as those reported here can be conducted, while sustaining long-term and critically accountable inquiry.
Data availability statement
The data and the code are available on GitHub via https://github.com/DHLab-Freiburg/premodern-slavic-narrative-analysis-pilot.
Acknowledgements
We thank Daniel Alcón Lopez (Digital Humanities Lab Freiburg), David Birnbaum (University of Pennsylvania), Hanne M. Eckhoff (University of Oxford), and Piroska Lendvai (Bavarian Academy of Sciences and Humanities) for valuable support. The usual disclaimers apply.
Author contributions
Analysis using GT data and philological interpretation: A.R.; Analysis using NLP methods: I.F.; Conceptualization: A.R., A.E., and I.F.; LLM-based experiments: A.E.; Methodology: A.R., A.E., and I.F.; Visualization: A.R., A.E., and I.F.; Writing: A.R., A.E., and I.F. Apart from our experiments with LLMs, we used LLMs for auxiliary tasks, such as copy-editing, formatting, or debugging. We did not use AI for main tasks, such as conceptualization or draft writing.
Competing interests
The authors declare none.
Ethical standards
The research meets all ethical guidelines, including adherence to the legal requirements of the study country.
Appendix
Translation table for network and bar plots
Table A1 Long description
Starting at the top-left, the table lists Old Church Slavonic tokens in the first column and their English translations in the second column. Each row contains two pairs: for example, the first row shows аѵрилиꙗнъ translated as Aurelianus and отьць as father. Subsequent rows include богъ as God, оуалъ as Valens, Василискъ as Basiliscus, Павьлъ as Paulus, воинъ as soldier, повелѣти as to order, воѥвода as voievode, правъ as right, вѣра as belief, правовѣрьнъ as true-believing, глаголати as to speak, пожещи as to burn, градъ as town, прити as to come, господь as lord, прѣподобьнъ as righteous, епискоупъ as bishop, рать as battle, жена as woman, рещи as to speak, жрьти as to sacrifice, сътворити as to create, жрьтва as sacrifice, свѧтъ as holy, житиѥ as life, тъгда as then, ити as to go, то as that, Исаакии as Isaacius, Христосъ as Christ, Иоулиани as Juliana, храмъ as temple, команъ as Comana (Cappadocia), хотѣти as to want, мъногъ as much or many, црькꙑ as church, молитва as prayer, цѣсарь as emperor, молити as to beg, чоудо as miracle, огнь as fire, Ѳеодосии as Theodosius. The table is structured with alternating pairs across each row, maintaining a consistent left-to-right, top-to-bottom order.
Consolidated clusters with roles, functions, relations, speech acts, focalization, certainty, and notes
Table A2 Long description
Beginning at the top row, the table contains ten clusters, each identified by a cluster ID and lemma. For clu:001, lemma мцс҇а, roles are martyr and intercessor, functions are sufferer for faith and performer of miracles, relations include relationship with God and believers, speech acts are prayer and plea for mercy, focalization is internal, certainty is medium, and notes describe the figure as a martyr involved in suffering, intercession, miracles, and divine communication from an internal perspective. Clu:002, lemma василискъ, roles are martyr and holy man, functions are intercessor and miracle worker, relations are with God and believers, speech acts are prayer and invocation, focalization is external, certainty is medium, and notes highlight his role as intercessor and miracle worker with external narrative perspective. Clu:003, lemma максимианоу, roles are emperor and persecutor, functions are ruler and antagonist, relations are opposition to Christians and command of subordinates, speech acts are commands and decrees, focalization is external, certainty is medium, and notes describe him as an emperor persecuting Christians with authoritative commands. Clu:004, lemma асклипиода, is a location serving as a setting for events involving military and religious figures, with external focalization, medium certainty, and notes indicating its role as a site for miracles and religious events. Clu:005, lemma агрипа, roles are persecutor and authority figure, functions are enforcer of pagan worship and opponent to Christian figures, relations are opposition to Василиск and service under Максимиан, speech acts are commands and questions, focalization is external, certainty is medium, and notes describe him as an antagonistic authority enforcing pagan worship. Clu:006, lemma госпожа, roles are believer and witness, functions are conversion and healing, relations are household and community, speech acts are praise and request, focalization is external, certainty is medium, and notes describe her as a witness to miracles leading to conversion and community praise. Clu:007, lemma христос, roles are healer and miracle worker, functions are performing miracles and healing, relations are worshipped by people and believed in by soldiers, no explicit speech acts, focalization is external from narrator, certainty is medium, and notes describe him as a figure performing miracles and healing. Clu:008, lemma воевода, roles are military leader and authority figure, functions are commands and interaction with subordinates, relations are with subordinates and religious figures, speech acts are commands and questions, focalization is external, certainty is medium, and notes describe him as a leader involved in decision-making and questioning. Clu:009, lemma богъ, roles are deity, creator, and sustainer, functions are receiver of worship, source of power, and judge, relations are worshipped by humans, opposed to idols, and interacts with saints, speech acts are commands, listening to prayers, and demonstrating power, focalization is omniscient and external, certainty is high, and notes describe him as the supreme deity with divine power and close relationship to saints. Clu:010, lemma мой, role is supplicant, functions are seeker of mercy and remembrance, relations are with a divine or higher power, speech acts are prayer and request, focalization is internal, certainty is medium, and notes describe a supplicant seeking mercy and remembrance in a personal plea.
Sentence-level evidence snippets per figure cluster (Arabic numbering)
Table A3 Long description
Starting from the top row, the table header contains three columns labeled cluster ID, lemma, and evidence snippets. The first data row lists clu:001, lemma мцс҇а, and three numbered Old Church Slavonic evidence snippets. The second row shows clu:002, lemma василискъ, and eight numbered snippets. The third row presents clu:003, lemma максимианоу, with one snippet. The fourth row displays clu:004, lemma асклипиода, and three snippets. The fifth row contains clu:005, lemma агрипа, with eight snippets. The sixth row has clu:006, lemma госпожа, and three snippets. The seventh row lists clu:007, lemma христос, with two snippets. The eighth row shows clu:008, lemma воевода, and two snippets. The ninth row presents clu:009, lemma богъ, with two snippets. The tenth row displays clu:010, lemma мой, and one snippet. Each evidence snippet is written in Old Church Slavonic and is numbered within its cell. The table is bordered horizontally and all columns are left-aligned.
Open access
Rapid Responses
No Rapid Responses have been published for this article.