Ignorance and AI-related finance

Recently, the Bank of England (BoE) has raised concerns about the rapid expansion of AI in the financial sector. A comprehensive Bank of England (2024) survey revealed that approximately 75% of financial companies are using AI technology. Sarah Breeden, the Bank’s deputy governor, warned that the widespread adoption of AI, including generative AI, introduces new and unpredictable risks to the financial system. The complexity and lack of transparency in AI decision-making make it difficult for financial professionals to fully understand and manage these risks. The survey found that only 34% of firms have a ‘complete understanding’ of the AI they use, while 46% have only a ‘partial understanding,’ leading to significant epistemic uncertainty. In response, Breeden emphasized that ‘managers of financial firms must ‘understand and manage what their AI models are doing’’ (Arnold, Reference Arnold2024).

These statements point to a central dilemma surrounding AI models in finance: the tension between the inherent and unavoidable ignorance in AI-driven finance – stemming from both human and algorithmic limitations – and the necessity to fill these epistemic gaps.

On this background, it would be useful to integrate ignorance studies (Smithson, Reference Smithson1989; Proctor and Schiebinger, Reference Proctor and Schiebinger2008; DeNicola, Reference DeNicola2017; McGoey, Reference McGoey2019; Gross and McGoey, Reference Gross and McGoey2023) with social studies on the adoption of AI, particularly machine learning (ML), in financial markets. The literature on automated finance has already emphasized the importance of ignorance in financial modeling (Svetlova, Reference Svetlova2018), for algorithm designers and traders (Lange, Reference Lange2016; Souleles, Reference Souleles2019; Borch, Reference Borch2022), as well as in financial regulation (Coombs, Reference Coombs2016) and in markets more general (Nik-Khah and Mirowski, Reference Nik-Khah, Mirowski, Beverungen, Mirowski and Nik-Khah2019). At the same time, recent research on AI has highlighted the significance of ignorance as a fundamental concept in AI-related contexts (Grill, Reference Grill2022; Wehrli et al., Reference Wehrli, Hertweck, Amirian, Glüge and Stadelmann2022; White and Lidskog, Reference White and Lidskog2022; Kirkegaard, Kristensen, and Lauridsen, Reference Kirkegaard, Kristensen and Lauridsen2023). However, the next logical step – integrating ignorance studies with the sociology of finance, particularly regarding the rise of AI and ML – has yet to be undertaken.

In ignorance studies, non-knowledge is considered as a social fact that can be analyzed through the lens of who does not know what, how they do not know it, and with what effect. These studies examine the absence or limitations of knowledge, as well as the deliberate construction of non-knowledge across various domains. Given that all participants in financial markets possess a certain degree of ignorance regarding AI technology – and that the technology itself is prone to ignorance – it is striking that AI-supported niches of finance have so far been overlooked as a relevant domain for ignorance studies. A more systematic research effort in this field is strongly needed.

This essay explores tentative directions of such research. We refer to the recent studies in sociology of finance which have emphasized the need to distinguish ML models from more ‘traditional’ theory-driven financial models, such as DCF, VaR, Black-Scholes, and CAPM, with respect to how they produce knowledge and ignorance (Spears and Hansen, Reference Spears, Hansen, Borch and Pardo-Guerra2025). In this essay, we investigate how the role and nature of theory and knowledge changes with AI-supported finance. More specifically, we ask how the roles of theory and causation have evolved in the forms of knowledge and ignorance now shaping financial practice, focusing on causal ignorance and the theory-agnostic nature of AI.

In addition to literature on the topic, the essay draws on 22 semi-structured interviews and informal conversations with finance industry professionals during participant observations at various industry conferences and exhibitions, totaling 75 hours. We interviewed finance professionals in various sectors (such as investments, commercial banking, credit services, central banking, and financial analysis) as we are interested in common patterns across sectors and aim to make a general point about ignorance in the financial sector. Depending on participants’ preferences, some interviews were audio-recorded and transcribed, while others were documented through detailed field notes, as certain participants declined recording. Both transcripts and notes were systematically coded to ensure consistency and analytical rigor.

We begin by exploring the various forms that ignorance can take in AI-driven finance, supporting our argument for the systematic sociological analysis of AI-related finance through the lens of ignorance studies. We then focus on a particular type of ignorance: causal ignorance. Here we argue that AI algorithms, including those accompanied by XAI methods, do not provide genuine causal explanations. Understanding causation is inherently theoretical, and for this reason, AI-driven finance remains marked by theoretical ignorance and can be seen as fundamentally theory-agnostic. Focusing on the relationship between theory and causation, we deliberate on how the ignorance generated by ML models differs from that associated with traditional models, and, more generally, how the role and nature of theoretical knowledge and ignorance in finance change. In particular, we examine empirical manifestations of theoretical ignorance by presenting three scenarios of how AI-driven finance engages with or distances itself from theory, evaluate these scenarios, and conclude with a discussion of findings and outlook for future research.

Pertinence and relevance of AI-related ignorance in financial markets

The integration of AI into finance introduces various forms of ignorance. In what follows, we provide a brief overview of some key forms and sources of ignorance in AI-driven finance. While this list is not exhaustive, we believe it offers a meaningful perspective on the fundamental epistemic challenges.

First, in today’s AI-based finance, as in many other contemporary knowledge intensive professional fields (Steinberg, Reference Steinberg2024; Schmidt, Putora, and Fijten, Reference Schmidt, Putora and Fijten2025), ‘knowledge is a function of market data’ (Borch, Reference Borch2022: 7). Resultantly, conventional knowledge risks associated with data (such as data quality in terms of relevance and accuracy) are heightened by the challenge that these risks may not be promptly and timely addressed by humans. Given the speed, complexity, and continuous learning capabilities of machines, human agents have limited capacity to identify flaws and faulty inferences related to data, despite the implementation of extensive data hygiene measures designed to prevent such issues. In particular, data scarcity is an important epistemic issue. Although this problem is not new (Israel et al., Reference Israel, Kelly and Moskowitz2020), one of our interviewees (Interview 15) particularly emphasized that finance in fact often relies on small datasets, typically comprising only a few hundred relevant observations, which might lead to misestimating the statistical power of AI models. He claimed that the shift from linear models like Value at Risk (VaR) to nonlinear machine-learning models exacerbates information uncertainty, making meaningful statistical validation increasingly difficult.

Second, there are many unknowns about how an AI algorithm will behave in situations for which it was not specifically trained. There is a concern that machine-generated knowledge may not adapt quickly enough in response to unexpected and severe contextual changes, such as central bank policy shifts or political turmoil. This is the problem of overfitting and model drift, where models’ behavior in untrained scenarios remains unknown. As a result, ‘algorithms – especially models derived directly from data via machine learning – […] are often so complex and opaque that even their designers cannot anticipate how they will behave in many situations’ (Kearns and Roth, Reference Kearns and Roth2020). Our interviewees specifically highlighted that AI systems’ knowledge is not contextual enough and exhibits severe gap from reality: AI models primarily learn from past data and cannot anticipate qualitative factors influencing company’s performance, such as the eminent departure of a key executive who has been a major contributor to company’s success. This is a crucial epistemic limitation in valuation tasks, for example. A financial consultant (Interview 13) stressed that many relevant factors missed by AI can be ‘seen’ – and thus help close the theory-practice gap – only in conversation with a client: To assess the company’s financial position for the next year, you have to discuss each balance sheet and income statement item with a client. Thus, while using AI, finance professionals are in doubt whether the model sees the whole picture and which aspects and changes of the world it is ignorant of.

The model drift just mentioned can easily occur also in a normal operational mode, that is, in routine situations where there are no significant external events of which the AI models are not aware. Even in such circumstances there may be small biases in the original training data that could be augmented over time as the models are exposed to new data. For that reason, standard AI governance guidelines stipulate continuous surveillance of AI models – something that is also empathized by our respondents. This includes the use of various formal tests (KS scores, PSI, etc.) and continuous assessments of the performance of AI output against given benchmarks. An interviewee mentioned that, in some cases, it is very clear when models are drifting but also emphasized that such drifts can happen very swiftly because AI models are ‘great accelerators’ (Interview 5). The same interviewee also stated that, in many instances, it is difficult to step in early enough to catch and prevent models from drifting, so that assessments in many cases have to be based on a combination of historical data and ‘experience’, or ‘domain knowledge’ (Interview 17, see also Hansen and Souleles, Reference Hansen and Souleles2023), thus emphasizing a subjective element to such assessments. Thus, AI designers and users often struggle to fully understand the dynamics that can lead to model drift or may not always recognize when drift occurs.

Thirdly, there is a risk of ‘not knowing what actually guides trading decisions’ (Borch, Reference Borch2022: 8). Some algorithmic investment decisions cannot be exhaustively described by codes or programs (e.g., deep neural networks); rather, there are hidden and complex mid layers of statistically trained algorithmic elements which are difficult to analyses due to their complexity. This problem is, of course, not restricted to AI in finance but is a general problem with AI. For this reason, there is a vast literature on explainability in AI (including in finance), and indeed also a literature on ignorance and AI, most of which is highlighting the opacity of AI systems (Emmert-Streib, Yli-Harja, and Dehmer, Reference Emmert-Streib, Yli-Harja and Dehmer2020; Boge, Reference Boge2022; Borch and Min, Reference Borch and Min2022; Khan and Vice, Reference Khan and Vice2022; Duede, Reference Duede2023; Carloni, Berti, and Colantonio, Reference Carloni, Berti and Colantonio2025; Schmidt et al., Reference Schmidt, Putora and Fijten2025).

Finally, there are unknown systemic and structural effects of AI use, also in finance. Severe, unpredictable market swings might occur, for example, if AI systems (autonomously) collude or herd. We received a foretaste of such developments during the ‘quant meltdown’ in August 2007, the flash crash on May 6, 2010, and the mini crashes in 2014 and 2020. Such fluctuations can jeopardize markets in new, unknown ways. AI applications might destabilize the financial system and make it more prone to crises in fundamentally unpredictable manners (Gensler and Bailey, Reference Gensler and Bailey2020; Svetlova, Reference Svetlova2022; Leitner et al., Reference Leitner, Singh, van der Kraaij and Zsámboki2024).

The examples provided illustrate that artificial intelligence in finance functions within various layers of ignorance, which include data limitations, model opacity, and the unknown systemic and structural effects. Financial markets are characterized by ‘symmetrical ignorance’ (Caves, Reference Caves2003: 75; Skidelsky, Reference Skidelsky2009: 45), meaning that all involved parties (including AI algorithms) are ‘unknowers’ (McGoey, Reference McGoey2019). While the discussed knowledge gaps have previously been addressed in various AI-related literature, the implications for social theorists have not been sufficiently or radically thought through. We will make suggestions in this direction by first discussing a particular type of ignorance in AI-driven finance, namely the lack of causal explanation, after which we will link this to lack of theoretical knowledge.

Ignorance and causation

As already mentioned, the opacity of AI models is one of the main reasons for the ignorance when it comes to use of and beliefs in AI-based knowledge. It is, for the reasons mentioned above, hard for humans to assess the validity of AI output, a problem which has sparked significant interest in explainable AI (Coeckelbergh, Reference Coeckelbergh2020; Emmert-Streib et al., Reference Emmert-Streib, Yli-Harja and Dehmer2020; Zodi, Reference Zodi2022). A significant part – although far from all – of that literature on explainability centers on the role of causation (Carloni et al., Reference Carloni, Berti and Colantonio2025). Most AI models are not only difficult to understand in regard to how they function generally. Their specific output, that is, which weights are attributed to different variables, how these variables may interact to cause specific output, etc. is also unclear. Therefore, it is argued that understanding the causal relationships between (data) input and model output will vastly improve the explainability of AI systems (Pearl, Reference Pearl2018b), leading scholars to coin the term causability (Holzinger et al., Reference Holzinger, Langs, Denk, Zatloukal and Müller2019; Chou et al., Reference Chou, Moreira, Bruza, Ouyang and Jorge2022; Carloni et al., Reference Carloni, Berti and Colantonio2025). Unlike standard explainability, which only lists the factors behind a prediction, causability clarifies the causal relationships between those factors and the outcome, making the explanation meaningful and actionable for human decision-making.

At this point, there are no convincing signs causability has been achieved in AI-supported finance; instead, we observe only partial steps in that direction. Some firms employ explainability tools such as SHAP, LIME, or feature importance to highlight which factors drive predictions (e.g., volatility, trading volume, sentiment). Interestingly, several of our interviewees claimed that they managed to effectively address the issue of AI explainability (e.g. Interviews 1, 10, 14, and 15), using various eXplainable AI (XAI) methodologies. Yet, these efforts do not lift ignorance of the type we just mentioned, while remaining correlation-based and falling short of uncovering causal mechanisms.

Often, XAI methods allow to identify which factors led, say, to holding a particular security position in a portfolio (e.g. price-to-earnings ratio of a target company) or to rejecting a loan application (e.g. income level of an applicant). Our interviewees highlighted that their AI systems might uncover new factors that seem to be relevant for the final decision but for an unknown reason. These factors might have ‘a completely non-intuitive relationship to the outcome’ (Interview 16): ‘for example, like, how long you’ve been on the job is a factor that these [credit default] models consider. But it’s not clear, like, whether you’ve been on the job too long or too short a time. Like, they just say the explanation would be, like, length of employment. And so, it’s very unclear, even from that statement, like, well, have I been here too long? Do they think that, like, you know, my prospects for, like, salary increase are, like, minimal because I’ve been in this position too long? Or have I been in too short a period of time? And so, they worry that I’m going to get fired. It’s, like, people [clients who get a rejection] have to, like, fill in all the details by imagining what the relationship could be.’ In other words, while XAI might disclose a factor that contributes to an outcome, the exact relationship between that factor and outcome (negative or positive effect, linear or nonlinear, etc.) may still be unclear.

Meanwhile, academic research on causal ML – drawing on Judea Pearl’s (Reference Pearl2018a; Reference Pearl2018b) framework, causal inference, and counterfactual methods – is making progress, but its uptake in financial institutions is still minimal, where predictive rather than explanatory uses of AI dominate. Thus, the recent efforts to incorporate XAI into financial decision-making cannot alleviate causal ignorance.

From causation to theory

Understanding the causal relationship which AI models estimate between data input and model output is, we suggest, intricately related to theoretical understanding. Theory (broadly defined) is about general and generalizable causal laws based on which predictions can be made (Felin and Holweg, Reference Felin and Holweg2024). We write ‘broadly defined’ because definitions of theory can be much narrower than that, ultimately including only formal mathematics-based theories such as the law of gravity (Emmert-Streib et al., Reference Emmert-Streib, Yli-Harja and Dehmer2020).

Theories understood broadly as scientifically based sets of propositions about causal relationships are often the basis upon which humans can understand and assess AI model output. For example, if and when the causal relationship an AI model estimates can be understood by humans, they will still need to understand the validity of that estimation. If an AI model links factor A with cause X, humans will still need to assess whether that estimated link is valid or whether it is caused by, say, biases in the model’s training data. Any such assessment of validity also brings questions of generalizability and boundary conditions – does the relationship only exist in specific contexts? That assessment will be based on existing empirical knowledge, perhaps even general experience, but it will also be based on theoretical generalizations. And reversely, the AI-based output should be relatable to theory, for example by enabling testing of theories (Desjardins-Proulx, Poisot, and Gravel, Reference Desjardins-Proulx, Poisot and Gravel2019; Mökander and Schroeder, Reference Mökander and Schroeder2022).

Importantly, such theory-based understanding of causal processes is not limited to scientists but extends to professionals more generally. Kurt Lewin’s famous quip about theories being practical of course also alludes to this. Theory is a key part of the heuristics and intuition’s of experts (Gobet and Chassy, Reference Gobet and Chassy2009), upon which their interpretations, understandings, and decisions are based. Theory is a basis for understanding why and how factor A relates to outcome X (Pearl, Reference Pearl2018a).

With the growing adoption of AI – and ML in particular – a new modeling culture is emerging in finance, one that is marked by causal, and thus theory-related, ignorance (Hansen, Reference Hansen2020; Borch and Min, Reference Borch and Min2022; Spears & Hansen, Reference Spears, Hansen, Borch and Pardo-Guerra2025; Millo, Spence and Xu, Reference Millo, Spence and Xu2024). Previously, sociology of finance examined in detail the intensive theoretical developments that accompanied the rise of key models such as MPT, CAPM, EMH, APT, VaR and BSM, which were implemented in practice through automated option trading, passive investing, portable alpha, factor investing, and other applications. During that period, finance became explicitly theory-driven (see Bernstein, Reference Bernstein1992, Reference Bernstein2007).

Although traditional finance models like Markowitz’s portfolio theory or the CAPM were recognized to be highly idealized, abstract, and unrealistic (e.g., assuming rational agents, frictionless markets, normally distributed returns). Yet, philosophers and sociologists argued that such models still generate knowledge: not by being literally true, but by highlighting causal mechanisms and offering heuristics for reasoning (Cartwright, Reference Cartwright1989; Mäki, Reference Mäki and Dilworth1992, Reference Mäki2009). For example, although Markowitz’s mean-variance portfolio model doesn’t describe how investors actually behave (they might not compute covariances), it provides a possible causal mechanism: diversification can reduce risk, and efficient portfolios balance risk and return. Moreover, sociologists of finance have examined how different unrealistic theory-based models have been used to shape, or even perform, financial reality as these models are connected to the complex dynamics of economic life through domain knowledge and human judgment (Mackenzie and Millo, Reference MacKenzie and Millo2003; Svetlova, Reference Svetlova2018).

Today, however, with rise of AI-based modeling, we observe that the search for sound decision-making in finance becomes less guided by theory. Instead, it is increasingly theory-agnostic, in the sense that decisions are now made on the basis of empirical correlations, predictive accuracy, and patterns detected in large datasets, rather than on causal explanations or theoretical models of how markets function.

Predictive ML algorithms are fundamentally ignorant of the world around them – they identify associations between variables, prioritizing predictive accuracy over understanding. This reflects a more general development in scientific knowledge production. In his well-known and highly controversial article, The End of Theory, Chris Anderson (Reference Anderson2008) argued that in an era dominated by algorithms, science no longer requires theory, explanations, or causality – correlations alone are sufficient. He quotes Peter Norvig, Google’s research director: ‘All models are wrong, and increasingly you can succeed without them’, clearly referring to theoretical models here. The focus shifts from understanding mechanisms and creating theories to simply acting on observed correlations. Thus, the rise of algorithmic methods challenges the roles of theory, causation, and in turn explanation.

As a result, theoretical ignorance seems indeed to be a critical factor in AI-driven segments of financial markets. But if financial markets become increasingly detached from theory due to the rise of AI, what implications will this have for the finance as an academic science and for behavior and decision-making of market participants? Asking that question simultaneously means asking whether the lack of theoretical knowledge challenges traditional scientific norms of explanation, causality, and generalizability, and whether the theoretical understanding of ‘why’ (Pearl, Reference Pearl2018a) is becoming progressively irrelevant or will be supplanted by alternative frameworks.

Emerging patterns in the relationship between AI models and financial theory

Given the exploratory nature of our study, we cannot fully answer the above questions regarding finance and theory, but we can provide a tentative outline. We identify three emerging patterns in finance: first, practices where AI models and theory are disconnected; second, cases where AI models are overridden by human domain knowledge; and third, instances where AI models are integrated with traditional financial theories.

Evaluating competing scenarios for the future of financial theory

Ultimately, our findings support the indications from research that financial modeling cultures are undergoing a fundamental transformation. From an ignorance studies perspective, this shift reflects a new form of epistemic structuring, where the role of various types of ignorance in AI-driven finance is also changing. Knowledge in financial ML-related modeling is increasingly built on pragmatic knowledge centered around data and algorithmic performance indicators such as accuracy, rather than causal explanations rooted in economic theory. The key challenge moving forward is to determine whether AI in finance will remain a theory-agnostic tool or whether financial professionals and researchers will find ways to integrate AI-driven insights into a broader theoretical framework.

We believe that the theory-agnostic practices (Scenario 1) will prevail. While the decoupling of AI-based knowledge and theory is not inevitable (we explored alternative scenarios), we believe that AI is likely to increase theoretical ignorance.

Scenarios related to bridging AI and financial practices (Scenario 2) or combining AI and theory (Scenario 3) are unlikely to crystallize other than in smaller, specific, domains. Firstly, the literature, as well as our interviews, suggest that the ML-practice gap might be too large to bridge effectively, creating challenges in aligning algorithmic decision-making with domain-specific expertise. Something similar also applies to theoretical integration where the span between AI-generated output and standard theoretical predictions may be too large to bridge.

In light of these challenges, the role of human expertise in ML-driven finance will likely evolve from interpreting model results through domain knowledge (which includes theoretical aspects) to organizing and facilitating the proper functioning of algorithms. Expertise will increasingly center on self-referential knowledge – knowledge structured around algorithm-related quality signals, such as accuracy performance (Millo et al., Reference Millo, Spence and Xu2024). In this process, one form of ML-related ignorance – data-related ignorance – receives attention (as data are cleaned, improved, and adjusted), while theoretical knowledge is increasingly neglected.

Note also that Scenarios 2 and 3 require a certain commitment. Overlay requires an active stance, questioning the AI-based output, while theoretical integration is time consuming and costly. Relying on AI output will in contrast be both fast and easy. A developer of an AI-based trading solution (Interview 18) made the observation, somewhat sarcastically, that if AI products were regulation compliant, this would in many cases be the excuse traders needed to rely heavily on the AI output. Compliance meant it was ok.

Finally, it should be mentioned that another tendency could be a shift toward simplicity. This can manifest itself in efforts to make ML models more intuitive and easier to understand (Hansen, Reference Hansen2020) or in finance professionals opting to avoid using ML algorithms for non-standardized, complex decisions (Interviews 11, 12, 13). For example, a bank employee responsible for technology integration to improve customer experiences across all available communication channels reported that his division – after initial attempts – abandoned creating AI-based credit decision-making models and ‘ended up with having […] rule-based systems of models’ so that ‘AI is not actually making the decisions … We are aiming for AI to work on the simplest of cases so that the person can work on the more complicated cases where somebody needs to think and reason more’ (Interview 11). In general, based on our interviews, it appears that finance professionals often prefer to use ML for simple, standardized tasks such as compliance checks, virtual assistance, industry comparisons, and automated sustainability reporting, in order to minimize potential ignorance-related issues.