When large language models (LLMs) are used for semantic data extraction from unstructured text, producing candidate relational facts from natural language, they may remain unreliable for tasks requiring complex combinatorial reasoning and global consistency. This paper proposes a logic-guided data extraction framework combining LLM-based extraction with answer set programming (ASP). The LLM produces candidate facts, whereas ASP performs validation, inference, consistency checking, and control. Unlike existing pipelines that query the LLM independently for all target predicates, the proposed approach uses ASP reasoning to identify which predicates are logically admissible at each stage and to guide extraction queries. By interleaving LLM calls with ASP derivation, the framework infers logically implied facts without further extraction and detects inconsistencies early. We formalize the pipeline and prove that, under mild assumptions, it is equivalent to the baseline approach with respect to the final extracted facts, while requiring fewer LLM calls. We also introduce a caching mechanism for logic-based control queries, exploiting monotonicity of conjunctive queries over incrementally constructed fact sets to reduce solver invocations. Experiments on ASP-derived benchmarks show that the framework reduces LLM calls and improves extraction quality by mitigating spurious outputs, demonstrating the value of non-monotonic logic programming for controlled semantic extraction.

Prolog is a well-known declarative programming language commonly used in introductory courses on logic and reasoning. However, many students find Prolog challenging because it lacks the familiar debugging mechanisms found in imperative languages. In large classes, this difficulty is exacerbated by the challenge of providing timely and personalized feedback to students. In this work, we introduce ProDebug, the first tool to combine large language models (LLMs) with spectrum-based and mutation-based techniques for automated debugging of Prolog assignments. ProDebug automatically identifies faults and proposes bug repairs for student Git submissions. Faults are detected using three approaches – spectrum-based, mutation-based, and LLM reasoning – while repairs are generated using mutation-based techniques and LLMs. Our evaluation on 1499 buggy student submissions from a bachelor’s level programming class demonstrates the potential of automated, LLM-augmented feedback systems to scale support for declarative programming education.

Natural language processing (NLP) has moved from a specialized research field into the everyday infrastructure of writing, search, translation, education, journalism, public administration, and scientific work. This transition changes what counts as progress. Accuracy, fluency, and benchmark performance remain important, but they are no longer sufficient when language technologies shape knowledge, decisions, identities, and public trust. This column introduces Responsible NLP as a research orientation that integrates fairness, transparency, privacy, safety, cultural diversity, environmental awareness, and human agency across the full life cycle of language technologies. It argues that responsibility is not an external constraint on innovation, but a condition for meaningful and trustworthy innovation. Future research must therefore ask not only whether an NLP system works but also for whom it works, under which assumptions, with what risks, and with what forms of accountability.

We investigate whether large language models (LLMs) threaten democracy through their persuasive capabilities. Using two survey experiments (N = 10,417) and real-world simulations, we compare the cost-effectiveness of LLM chatbots against traditional campaign tactics, taking into account both the “receive” and “accept” steps in the persuasion process. Our design advances prior research by assessing extended human-LLM interactions and measuring short- and long-term effects across three political domains. We find that while LLMs are comparably persuasive to campaign ads once seen, real-world impact depends on both message reception and acceptance. Simulations estimate LLM-based persuasion costs $48–$75 per voter versus $100 for traditional methods. However, traditional methods currently scale more effectively. While LLMs do not yet offer substantially greater potential for large-scale persuasion, this may shift as capabilities improve and techniques for scalable exposure become feasible.

This study addresses the challenges of performing narratological analysis on low-resource languages, with a focus on Old Church Slavonic. Understanding the roles, interactions, and networks of persons is central to narrative analysis, yet such investigation is hindered by the scarcity of experts and the limited availability of annotated resources. We explore both established natural language processing (NLP) methods and large language models (LLMs) for analyzing pre-modern Slavic Lives of Saints, including several Slavic versions, the Greek original, and an English translation. Pre-modern Slavic texts pose particular difficulties due to rich morphology, orthographic variation, and limited standardization, which complicate the application of both traditional NLP tools and off-the-shelf LLMs. Through experiments using annotated and non-annotated ground truth data, we demonstrate that while conventional NLP methods often reach their limits on such low-resource, highly variable texts, LLMs provide complementary capabilities that can support narratological insights, especially in tracking persons and their interactions, albeit with important caveats regarding accuracy and coverage.

Web-enabled large language models (LLMs) frequently answer queries without crediting the web pages they consume, creating an “attribution gap” in responsible artificial intelligence (AI) usage—defined as the difference between relevant URLs read and those actually cited. Drawing on approximately 14,000 real-world LMArena conversation logs with search-enabled LLM systems, we document three exploitation patterns: (1) no search: 34% of Google Gemini and 24% of OpenAI GPT-4o responses are generated without explicitly fetching any online content; (2) no citation: Gemini provides no clickable citation source in 92% of answers; (3) high-volume, low-credit: Perplexity’s Sonar visits approximately 10 relevant pages per query but cites only three to four. A negative binomial hurdle model shows that the average query answered by Gemini or Sonar leaves about three relevant websites uncited, whereas GPT-4o’s tiny uncited gap is best explained by its selective log disclosures rather than by better attribution. Citation efficiency—extra citations provided per additional relevant web page visited—varies widely across models, from 0.19 to 0.45 on identical queries, underscoring that retrieval design, not technical limits, shapes ecosystem impact. To advance auditing and monitoring of AI systems, we recommend a transparent LLM search architecture based on standardized telemetry and full disclosure of search traces and citation logs.

This paper evaluates the performance of baseline and domain-augmented ChatGPT models for literature-based knowledge support in flood susceptibility mapping (FSM) using machine Learning approaches. To assess this, we designed five key questions related to FSM, with benchmark responses derived from our comprehensive review article (Pourzangbar et al., Journal of Flood Risk Management 18, e70042), which analyzed 100 studies on ML applications in FSM. The same questions were posed (i) to standard ChatGPT-4 and ChatGPT-4o models without additional contextual material, and (ii) to a domain-augmented GPT-4 configuration (Chat-FSM) equipped with retrieval access to the 100 reviewed articles. The comparison highlights that GPT-based models can reasonably reproduce frequently reported machine learning models and conditioning factors from the reviewed literature, but show weaker consistency in feature selection methods, often suggesting less relevant techniques. Among the models, ChatGPT-4o demonstrated the weakest alignment with benchmark data, while Chat-FSM demonstrated the highest agreement with the benchmark dataset across most evaluated questions. In terms of application-level efficiency, GPT models required substantially less time and computational effort compared to manual literature synthesis under the defined experimental setup. While ChatGPT-based systems can support literature-informed exploration in FSM, human expertise remains essential for critical reasoning, methodological design, and application to novel or context-specific scenarios.

Traditional perceptual models are ill-equipped for the high-dimensional data, such as text embeddings, central to modern psychology and AI. We introduce the double machine learning lens model, a framework that utilizes machine learning to handle such data. We applied this model to analyze how a modern AI and human perceivers judge social class from 9,513 aspirational essays written by 11-year-olds in 1969. A systematic comparison of 45 analytical approaches revealed that regularized linear models using dimensionality-reduced language embeddings significantly outperformed traditional dictionary-based methods and more complex non-linear models. Our top model accurately predicted human $(R^{2}_{CV} =0.61)$ and AI $(R^{2}_{CV} =0.56)$ social class perceptions, capturing over 85% of the total accuracy. These results suggest that “unmodeled knowledge” in perception may be an artifact of insufficient measurement tools rather than an unmeasurable intuitive process. We find that both AI and humans use many of the same textual cues (e.g., grammar, occupations, and cultural activities), only a subset of which are valid. Both appear to amplify subtle, real-world patterns into powerful, yet potentially discriminatory heuristics, where a small difference in actual social class creates a large difference in perception.

Interest in large language models (LLMs) as a tool for meta-analyses and systematic reviews (MA/SRs) is growing. We prospectively developed 515 unique prompts by predefined screening-related categories and tested with open-access LLMs (Llama, Mistral) against four gold-standard MA/SRs from different medical fields published after the LLMs’ training cut-offs, using a Python-based pipeline. Heterogeneity between prompts was quantified, and hypothetical workload/cost reduction with top-performing prompts calculated. Across 12,360 pipeline runs, LLMs versus MA/SRs reached average recall/sensitivity = 83.6 ± 17.0%, precision = 18.5 ± 15.6%, specificity = 36.6 ± 23.7% F1-score = 27.6 ± 17.2%, and accuracy = 61.1 ± 11.0%. F1-scores were significantly higher when prompts focused on methods (0.78 ± 0.40%), explicitly mentioned MA/SR screening (0.81 ± 0.37%), included the comparison MA/SR’s title (5.64 ± 0.37%) or selection criteria (8.05 ± 0.68%), and with more LLM parameters (70b = 4.48 ± 0.31%, 123b = 7.77 ± 0.31%), but lower when screening abstracts instead of titles (−3.67 ± 0.28%). In LLM-base preselection, top-performing F1-score prompts (recall/sensitivity = 72.2%, specificity = 66.1%, precision = 28.6%) would reduce screening demands by 34.5%−37.5%, saving 8.4–8.8 weeks of work and 17,592–18,552. Recall/sensitivity increased with less MA/SR information contrasting F1-score results, which highlights a recall/sensitivity-precision/specificity trade-off. F1-score increased with detailed MA/SR information, while recall/sensitivity increased with shorter, zeroshot prompts. We provide the first prospectively assessed prompt engineering framework for early-stage LLM-based paper screening across medical fields. The publicly available Python pipeline and full prompt list used here support further development of LLM-based evidence synthesis.

This article examines the transformative impact of large language models (LLMs) on online content moderation, revealing a critical gap between platforms’ rule-based policies and their AI-driven enforcement mechanisms. Using Facebook’s hate speech moderation policies and practices as a case study, we identify a paradox: while content policies are increasingly rule-oriented, AI-driven enforcement seems to operate in a standard-like manner. This disconnect creates transparency, consistency and accountability challenges relating to the delineation of online freedom of expression that are not addressed in the literature, and require attention and mitigation. In this specific context, we introduce the concept of ‘rules by the millions’ to describe how AI systems actually operate through generating vast networks of micro-rules that evade traditional regulatory oversight. This phenomenon disrupts the conventional rules-versus-standards framework used in legal theory, raising urgent questions about the adequacy of current AI governance mechanisms. Indeed, the rapid adoption of LLMs in content moderation has outpaced the human capacity to monitor them, creating a pressing need for adaptive frameworks capable of managing the evolving capacities of AI.

This study investigates the use of large language models (LLMs) to classify question utterances within verbal design protocols according to Eris’ (2004) taxonomy. We evaluate the performance of two proprietary LLMs – OpenAI’s GPT-4.1 and Anthropic’s Claude Sonnet 4.5 – across experiments designed to assess classification accuracy, sensitivity to prompt configuration and in-context learning (ICL), and generalization across datasets and models. Using two human-coded datasets of differing size and quality, we measure alignment between LLM-generated labels and human judgments at both question category and subcategory levels. Results show that both LLMs achieved moderate to strong alignment rates at the category level (up to 85.7% for GPT-4.1 and 82.9% for Claude Sonnet 4.5), with substantially lower alignment at the more granular subcategory level. Performance differences across prompt configurations and ICL conditions were small, indicating robust generalization across datasets and transferability of prompt designs. While these results suggest that LLMs can effectively support scalable question classification, human judgment and oversight remain essential. Future research should explore the development and evaluation of alternative hybrid human–LLM workflows in protocol analysis, as well as the use of smaller or open-source models to address data privacy concerns.