Plain language summary

When a cancer patient is dismissed because an algorithm misread her fatigue as cognitive decline, the problem is not only technical – it is communicative. This article examines how AI systems in medicine and work can misinterpret human behavior, and why feedback is the missing safeguard.

This article introduces the idea of double noise: when human judgment, which is variable, and algorithmic judgment, which is often simplified, overlap and reinforce each other. The result can be unfair or unsafe outcomes. Drawing on narrative theory, the article also shows how these systems do not merely misread people – they actively construct false stories by imposing coherence on incomplete signals, stories that then resist correction because no feedback channel exists to revise them.

The analysis draws on two case studies. The first is a fictional story about Leo, a worker whose illness led to short-term changes in her digital activity. Corporate dashboards misread these temporary signals as lasting decline, and her employer dismissed her without asking for context. Though fictional, this scenario is increasingly plausible in workplaces governed by algorithmic monitoring. The second examines a synthetic dataset on stroke prediction, designed for teaching. It illustrates how routine design choices – such as class balancing or metric selection – can obscure critical errors, especially false negatives.

Both cases show how, without feedback, small distortions harden into entrenched misjudgments. The article argues for systems designed to keep interpretive channels open: where professionals can question algorithmic outputs and individuals can challenge decisions that affect their health, work, or dignity. The framework developed here is intended as a transferable analytical tool – applicable wherever AI systems configure heterogeneous signals into consequential stories, suppressing the feedback that would allow those stories to be told differently.

Emotion, misrecognition, and epistemic noise

In recent years, there has been a marked increase in attempts to quantify emotional states using biometric and behavioral markers. Many contemporary AI systems still rely on reductionist theories, most notably those influenced by Ekman’s model of “basic emotions” (Ekman Reference Ekman1992; Ekman and Friesen Reference Ekman and Friesen1969), which treat affect as a set of universal, biologically hardwired categories. Much of affective computing depends on corpora annotated with Ekman’s categories – happiness, sadness, anger, fear, surprise, and disgust – despite decades of critique and evidence of cultural variability. Lisa Feldman Barrett offers one of the most influential counterarguments, insisting that there are no universal “fingerprints” of emotion. Instead, emotions are socially and culturally constructed categories that emerge from context-dependent interpretations of bodily signals (Barrett Reference Barrett2017). This “variety is the norm” poses a major challenge for algorithmic systems, which continue to treat emotions as static labels tied to physiological or behavioral patterns.

This conceptual mismatch is not only epistemological but also narratological. A system trained on Ekman’s universal categories functions as an unreliable narrator in the precise sense theorized by Nünning (Reference Nünning, Grünzweig and Solbach1999): not a narrator who distorts knowingly, but one whose epistemic architecture prevents it from recognizing its own distortion. It produces a story about the subject’s affective state – one the subject herself would recognize as false – without any capacity to register the gap between its account and the lived reality it claims to represent. This is cognitively restricted narration in its most structural form: the model cannot know what it cannot see, and what it cannot see is precisely what Barrett shows to be constitutive of emotional meaning.

This unreliability is compounded by a problem of focalization. Ekman-based systems operate under the illusion of zero focalization (Genette Reference Genette1980): they claim to capture the subject’s full affective profile from facial micro-expressions, vocal modulation, or interaction rhythms, projecting a totalizing epistemic authority that their actual access to surface proxies could never justify. In Genette’s terms, this constitutes a structural paralepsis: the system gives more interpretive weight to its output than its restricted focal position authorizes. What appears as objective affect recognition is in practice a severely constrained point of view masquerading as total knowledge.

The result is also a failure of situatedness (Herman Reference Herman2009). Barrett’s central claim – that emotions are context-dependent interpretations of bodily signals – is, in narratological terms, a claim about situatedness: emotional meaning cannot be detached from the specific embodied, relational, and cultural occasion that produces it. A model trained on decontextualized corpora treats as context-free what is irreducibly situated, stripping away the discourse context that would be necessary for meaningful interpretation. When Leo’s fatigue is read as disengagement, or her silence as unreliability, the system is not merely misclassifying – it is producing a desituated narrative that forecloses the very context that would make her signals legible.

Finally, and most consequentially, this is a failure of emplotment (Ricoeur Reference Ricoeur1984). Where narrative configuration should synthesize heterogeneous affective signals into a meaningful and revisable whole, Ekman-based systems freeze discontinuous and culturally variable expressions into single, irrevocable categorical verdicts. What emplotment, properly understood, keeps open – the possibility of renarrating, of integrating new context into a revised account – the algorithmic system forecloses. A momentary shift in vocal tone becomes “anger”; a brief absence of eye contact becomes “disengagement”; a two-week fluctuation in productivity becomes “cognitive decline.” Each of these moves enacts the same narrative violence: the compression of a living, revisable signal into an irreversible story.

This conceptual mismatch produces epistemic noise: a systematic gap between the richness of lived affective experience and the reductive formats through which it is captured and classified. When algorithmic simplifications then meet uncritical institutional deference, the result is double noise: epistemic variability in human affect flattened into algorithmically rigid categories, and those categories cemented into definitive judgments without contestation. What is missing is not only accuracy but also feedback – the channel through which subjects could recontextualize how their emotional states are inferred, and through which errors could be re-situated in lived experience rather than hardening into institutionalized misrecognition.

The EU Artificial Intelligence Act (2024) (European Parliament and Council of the European Union (EU) 2024) explicitly prohibits the use of emotion recognition in workplaces and education. Yet this safeguard addresses only direct recognition, not the indirect reconstruction of affective states through behavioral proxies. Typing speed, login times, geolocation traces, or email responsiveness can all serve as soft indicators of cognitive or affective states, enabling what McStay (Reference McStay2018) terms “emotional AI.” In such cases, emotion is not formally detected but statistically inferred, and decisions are taken based on deviations from behavioral norms. The prohibition is thus respected in form but circumvented in practice – and the narratological failures of unreliable narration, false focalization, desituatedness, and foreclosed emplotment operate undisturbed beneath the regulatory threshold.

In Leo’s case, this also amounted to cultural injustice. Coming from an Asian background where collective harmony is valued over individual disclosure, her silence was not simply a personal choice but a culturally situated communicative act – one that the system’s predefined taxonomies rendered illegible. As Fricker (Reference Fricker2007) argues, such dynamics constitute epistemic injustice: the systematic exclusion of certain voices from credibility. In more individualist contexts, she might have taken sick leave without penalty; her decision to keep working – shaped by collective norms of discretion – was reinterpreted as unreliability, sealing her exclusion. Feedback could have reopened the communicative loop; its absence transformed epistemic noise into institutionalized misrecognition.

Between prediction and explanation: Revisiting the “broken leg”

If anything has marked the history of research on human judgment, it has been a constant tension between statistical prediction and human intuition. In the 1950s, Meehl (Reference Meehl1954) famously demonstrated that actuarial models consistently outperformed clinical experts in predicting outcomes, such as academic success or psychiatric relapse. Yet Meehl himself acknowledged a crucial limitation, illustrated by what became known as the “broken leg case.” Suppose a man usually goes to the movies every Tuesday, and a statistical formula predicts his attendance. If we learn, however, that he has a broken leg, this singular, context-rich event – though statistically rare – completely invalidates the prediction. The lesson: even highly accurate models can fail spectacularly when faced with anomalies that resist probabilistic encoding.

In clinical AI, these anomalies are not hypothetical, but rather close to the norm. Patients with rare diseases, comorbidities, or atypical trajectories often fall outside the training distribution of predictive systems. These cases, which can be considered by analogy as “broken leg” cases, are frequently treated as outliers or noise and removed during preprocessing in the name of model optimization. Yet they may represent precisely those patients most in need of medical care. What appears to be a statistical inconvenience is, in fact, an ethical, social, and clinical failure: the tension between model generalization and context specificity resurfaces not as an epistemological footnote, but as a central challenge to justice in healthcare.

Recent work in machine learning has underscored similar concerns. Researchers in anomaly detection argue that discarding rare or unusual cases can distort both training and evaluation, especially in medical contexts, where anomalies may carry decisive clinical meaning. Others highlight how anomaly detection and outlier exposure techniques could improve robustness by explicitly training models to recognize and flag such cases rather than silently misclassifying them (Hendrycks, Mazeika, and Dietterich Reference Hendrycks, Mazeika and Dietterich2019; Machado, Silva, and Gama Reference Machado, Silva and Gama2023). These approaches reveal a structural analog to the “broken leg case”: outliers are not statistical curiosities but potential signals of high-risk conditions.

Leo’s story embodies one such “broken leg” case. Her temporary changes in digital behavior, caused by the embodied toll of cancer treatment, fell entirely outside the expectations of workplace algorithms. The system recoded her suffering into a misleading narrative of cognitive decline, rendering her condition illegible within dominant predictive frameworks. This is precisely the kind of anomaly that cannot be anticipated through probabilistic reasoning but demands contextual interpretation. In Herman’s terms, this constitutes a failure of situatedness: the system lacks access to the embodied and biographical discourse context that would make the anomaly legible. In Leo’s case, the absence of feedback made such interpretation impossible: her lived signals were treated as noise, erased from the decision process.

This unresolved conflict sets the stage for the next case: a pedagogical analysis of stroke prediction using a widely circulated synthetic dataset. Unlike Leo’s story, where the anomaly was biographical and context-specific, the stroke dataset illustrates how anomalies can be structural, produced not by patients themselves but by modeling conventions. The dataset was originally designed for teaching programming and machine learning practices, not for clinical deployment. Its purpose was educational, yet the way it is typically used – in exercises on classification, clustering, and model evaluation – reveals how routine design choices, such as data balancing, metric selection, or threshold setting, can systematically obscure false negatives. What is sidelined for didactic neatness are precisely the patients who, in a real-world setting, would be at highest risk. In this sense, the stroke dataset functions as a “broken leg case” by design: its pedagogical simplifications intentionally strip away clinical nuance, creating blind spots that mirror real-world failures of predictive systems.

Crucially, the “broken leg case” underscores that predictive failures are not only statistical but also communicative. When anomalies – whether lived or structural – are treated as noise, the signal of a patient’s condition is lost in translation. What fails is not simply the model’s accuracy, but the chain through which information is encoded, transmitted, and decoded. Without a channel for feedback, these signals cannot re-enter the communicative loop; they are silenced before they can challenge the authority of the model.

The broken leg case is, in narratological terms, not only an epistemological failure but also a failure of focalization. The actuarial model operates under the same illusion of zero focalization that governs Leo’s corporate dashboard: it projects a totalizing epistemic authority – the man will go to the cinema on Tuesday – from a severely restricted focal position that cannot accommodate what it cannot see. The broken leg is precisely the kind of embodied, context-rich signal that resists probabilistic encoding, and its invisibility to the model constitutes a structural paralepsis: the system gives more interpretive weight to its prediction than its actual epistemic access could ever authorize. This is also a failure of emplotment (Ricoeur Reference Ricoeur1984): the model freezes a dynamic, revisable behavioral pattern – Tuesday cinema attendance – into an irrevocable causal story that forecloses the very anomaly that would make it false. What Meehl’s case reveals, then, is not merely a statistical limit but a narrative one: predictive systems fail not only when their data are incomplete, but also when their architecture cannot renarrate in the face of lived reality.

Leo’s case is, in Meehl’s terms, a “broken leg” case: a singular, context-rich anomaly that invalidates the prediction not because the model is technically deficient, but because its epistemic architecture – operating under the illusion of zero focalization – cannot accommodate what it cannot see. The absence of feedback is what prevents the broken leg from re-entering the communicative chain – and what allows a failed emplotment to solidify into verdict. The stroke dataset that follows enacts the same logic at a structural level: here, the anomaly is not biographical but methodological, produced not by a patient’s hidden condition but by the modeling conventions themselves. In both cases, what disappears is not noise in the technical sense – it is the signal that matters most.

What is missing, then, in both Leo’s dismissal and in the pedagogical treatment of stroke prediction, is a robust mechanism of feedback. Feedback would not erase uncertainty, but it would allow context to interrupt prediction, giving rare conditions and exceptional cases the possibility of recognition. This move from descriptive model to normative claim is not arbitrary: once communication is understood as the medium through which clinical judgment is formed, the conditions that distort or foreclose it become ethical stakes, not merely technical ones.

Shared judgment and the ethics of feedback

The key issue in algorithmic and cognitive noise is not merely one of accuracy, but of communication: as information travels through systems of judgment, some signals are amplified while others are deleted, ignored, displaced, or mistranslated. What disappears in this process is the patient’s living signal, the embodied reality that resists neat encoding. Noise, in this sense, is not only technical but also cultural: signals may be dismissed because they do not align with normative expectations of productivity, compliance, or disclosure.

To grasp how such distortions arise and persist, this section turns to the Shannon–Weaver model of communication (Shannon and Weaver Reference Shannon and Weaver1949). Originally conceived for telegraph lines and technical signal transmission, the model offers a surprisingly apt perspective for examining AI-supported decision-making. It reminds us that every prediction is also a message: something must be encoded, transmitted, and decoded, always under the threat of noise. Noise, in this sense, is not a minor glitch but a structural form of uncertainty that, if ignored, can open the way to epistemic injustice.

The model distinguishes six components – source, encoder, channel, noise, decoder, and destination – each a potential fracture point where meaning may shift. Later refinements by Schramm (Reference Schramm and Schramm1954) and Barnlund (Reference Barnlund and David Mortensen1970) introduced feedback as the element that transforms linear transmission into dialogue. In clinical AI, feedback is not a decorative feature; it is the moment where signals are tested against lived knowledge, where patients and practitioners can resist, correct, or reinterpret algorithmic claims. Without such loops, distortions solidify into decisions, producing unfair norms that become increasingly difficult to contest. Feedback is therefore not only a safeguard against error but also a mechanism that clarifies responsibility, redistributing accountability across patients, practitioners, and systems.

Seen from this perspective, noise is not merely an error to be filtered out. When algorithmic variability and human bias converge without the corrective force of feedback, noise does not diminish – it doubles. This phenomenon of double noise explains why failures persist, why judgments misalign, and why epistemic injustice can become entrenched in clinical AI.

Figure 1 reinterprets the Shannon–Weaver model for clinical AI, illustrating how algorithmic and epistemic noise can propagate through each stage when feedback is absent or suppressed.

Figure 1.

Reinterpretation of the Shannon–Weaver model in the context of clinical AI. Double noise propagates across multiple stages when feedback is weak or absent.

Mapping these distortions across the communicative chain reveals that noise does not accumulate randomly – it concentrates at the stages where human judgment and algorithmic output intersect without mediation. Table 1 illustrates this pattern, showing how algorithmic and cognitive noise converge at each stage when feedback is absent. Unlike other stages, feedback is transversal: it can retroactively alter interpretation across the entire chain.

Table 1.

Sources of algorithmic and cognitive noise across the Shannon–Weaver model in clinical AI

Table 1 Long description

The table consists of three columns: Stage, Algorithmic noise, and Cognitive noise.

* Information source: Algorithmic noise includes biased, incomplete, or non-representative training data. Cognitive noise includes selective attention, prior assumptions, and limited anamnesis.

* Encoder: Algorithmic noise includes faulty sensors, data transformation errors, and preprocessing artifacts. Cognitive noise includes inconsistent documentation and ambiguous clinical notes.

* Channel: Algorithmic noise includes transmission delays, interface mismatches, and system latency. Cognitive noise includes fatigue, cognitive overload, and environmental distractions.

* Noise source: Algorithmic noise includes class imbalance, adversarial examples, and model variance. Cognitive noise includes stress, time pressure, implicit bias, and contextual misalignment.

* Decoder: Algorithmic noise includes non-interpretable outputs and instability in predictions. Cognitive noise includes framing effects, misinterpretation of probabilities, and reliance on heuristics.

* Destination: Algorithmic noise includes misclassification due to poor generalization and interface confusion. Cognitive noise includes overreliance on system outputs, mistrust, and interpretive shortcuts.

* Feedback: Algorithmic noise includes neglected feedback signals, retraining misalignment, and audit failure. Cognitive noise includes suppression of dissent, epistemic deference, and lack of critical reflection.

Seen in clinical practice, each element of the Shannon–Weaver chain has a direct analog. The information source may be a patient history or a clinical trial dataset; the encoder a diagnostic test or algorithmic feature extractor; the channel an electronic health record system; and the decoder a clinician interpreting model outputs. At each stage, algorithmic and cognitive noise intertwine, with feedback loops determining whether errors are corrected or silently reinforced. This distributed framing shifts the question of accountability: failures do not belong to a single faulty model or a careless clinician, but to the fragile links between them. If the communicative chain is where meaning is formed – and where it can be lost – then the ethical imperative is not to optimize individual nodes but to protect the integrity of the chain as a whole. As Mittelstadt et al. (Reference Mittelstadt, Allo, Taddeo, Wachter and Floridi2016) and Ananny and Crawford (Reference Ananny and Crawford2018) emphasize, governance failures often arise not from the absence of data but from the breakdown of communication and accountability.

Clinical AI, therefore, must be designed for shared judgment – a mode of decision-making in which clinicians, patients, and algorithmic systems participate as co-interpreters rather than as isolated producers and consumers of scores. Unlike fixed decision thresholds, which cannot account for context-specific anomalies, shared judgment institutionalizes the conditions under which uncertainty can be named, negotiated, and revised. In narratological terms, it restores what Herman calls situatedness (Herman Reference Herman2009) – the feedback channels through which a system’s output can be reinterpreted in light of the specific discourse context that produced it.

This requires embedding iterative feedback that enables contestation and recalibration across the entire communicative chain. The ethic that follows is one of feedback ethics: systems that resist premature closure, foreground plurality, and institutionalize opportunities for negotiation (Doshi-Velez and Kim Reference Doshi-Velez and Kim2017; Fricker Reference Fricker2007). This normative move is not external to the Shannon–Weaver model but internal to it: once communication is understood as the medium through which clinical judgment is formed, the conditions that distort or foreclose it are not merely technical failures but ethical ones. The case study that follows makes this concrete.

With this theoretical framework established, the article now turns to a technical case study on stroke prediction. This case illustrates how communicative fractures identified in the Shannon–Weaver model materialize in practice: routine preprocessing steps and metric choices, once seen as neutral, propagate double noise and foreclose feedback.

Double noise and blind spots in stroke prediction

To complement this narrative, the present section turns to a technical case study: the analysis of a pedagogical dataset for stroke prediction. Although synthetic and simplified, this dataset cannot function as a direct proxy for real-world clinical deployment. Its value is strictly pedagogical: it demonstrates how structural blind spots emerge from routine modeling conventions, not that such blind spots occur with this frequency or severity in clinical practice. The generalization it supports is therefore methodological rather than epidemiological – it shows how erasure can be normalized by default pipelines, not how often this occurs in deployed systems.

This distinction matters: the ethical argument does not depend on the dataset’s clinical representativeness, but on the structural logic it makes visible. Importantly, this dataset was never intended for clinical use but for teaching programming, with a special focus on false positives. Its educational intent highlights a paradox: even datasets created for pedagogical clarity can reproduce structural blind spots when feedback is absent. Tracing this logic requires applying the same narratological concepts introduced in the preceding sections – unreliable narration, focalization, paralepsis, and emplotment – to the pipeline itself: not as metaphors imported from literary theory, but as analytical tools that make visible the communicative failures that aggregate metrics routinely conceal.

Supervised models: Results and critical analysis

A Random Forest classifier was trained with default settings, except for a maximum tree depth of 3 to preserve interpretability and reduce overfitting. Stratified sampling maintained the original class distribution.

Overall performance appeared excellent – 95.4 percent accuracy and a weighted F₂ score of 94.5 percent. Yet the model failed to identify a single stroke case in the test set. All positive cases were misclassified as negative (Figures 2 and 3).

Figure 2.

Class imbalance in the stroke dataset. The vast majority of patients are labeled as No Stroke, while only a small minority experienced a stroke. This imbalance inflates accuracy and conceals failures in detecting critical cases.

Figure 3.

Confusion matrix of the Random Forest model. All stroke cases in the test set were misclassified as non-stroke, illustrating a total failure to detect the minority class.

Performance summary:

• • Accuracy: 95.4 percent.

• • Weighted precision: 90.9 percent.

• • Weighted recall: 95.4 percent.

• • F₂ score: 94.5 percent.

This failure is not only statistical but also narratological. The Random Forest pipeline operates, like Leo’s corporate dashboard, under the illusion of zero focalization: it produces a verdict of “no stroke” with the same epistemic authority that the dashboard projected “cognitive decline,” while both systems lack access to the very information that would make their outputs meaningful. In Genette’s terms, this constitutes a structural paralepsis at the level of the model itself: the classifier gives more interpretive weight to its output than its restricted epistemic position – trained on imbalanced proxies, evaluated on aggregate metrics – could ever authorize. The pipeline is, in this sense, an unreliable narrator: it produces a story of clinical safety that the patients it misclassifies would recognize as false, not through duplicity but through structural ignorance of what it cannot see. The absence of feedback forecloses the possibility of renarration: once the model outputs “no stroke,” no channel exists through which the false negative can re-enter the communicative chain as a signal worth interpreting.

These numbers conceal the clinical failure: the model “succeeds” by correctly classifying the majority class while systematically ignoring the minority. This behavior is well known in the literature on imbalanced learning, which shows that class weighting, oversampling, or threshold adjustment alone rarely resolve the deeper problem of underrepresented cases (Chawla et al. Reference Chawla, Bowyer, Hall and Philip Kegelmeyer2002; He and Garcia Reference He and Garcia2009). Even when these remedies improve numerical balance, they do not address the ethical implications of systematically missing rare but clinically decisive cases. To contextualize this failure, a counter-scenario was applied: a Logistic Regression model trained on SMOTE-balanced data. The results illustrate the trade-off that “normalize erasure” conceals. With SMOTE, recall for the minority class improved substantially – 42 of 71 stroke cases were correctly identified, compared to zero in the default pipeline. However, this gain came at significant cost: accuracy dropped from 95.4 percent to 68.3 percent, and false positives increased to 457. This contrast demonstrates that the claim of “normalization of erasure” is not metaphorical but empirically grounded: the default pipeline does not merely underperform – it systematically renders the minority class invisible while projecting an illusion of reliability.

Moreover, standard evaluation metrics can reinforce this illusion of success. As Saito and Rehmsmeier (Reference Saito and Rehmsmeier2015) demonstrate, accuracy and ROC curves are misleading under severe class imbalance, whereas precision–recall analysis provides a more informative view of minority-class performance. In a clinical context, such blind spots could mean missed interventions, increased morbidity, or preventable deaths. As Hooker et al. (Reference Hooker, Moorosi, Clark, Bengio and Denton2021) argue, framing bias solely as a “data problem” ignores how evaluation pipelines themselves sustain systemic failures. Pedagogical datasets exacerbate this by teaching students that high accuracy is sufficient, reinforcing illusions of reliability. Similarly, Ghassemi, Oakden-Rayner, and Beam (Reference Ghassemi, Oakden-Rayner and Beam2021) caution against the false hope that explainability or transparency alone will resolve these failures, stressing that deeper attention to evaluation practices and error distributions is needed.

Unsupervised clustering and latent profiles

To probe heterogeneity beyond binary classification, unsupervised clustering was applied using K-means on normalized features: age, BMI, and avg_glucose_level. After excluding rows with missing values, the Elbow Method indicated an optimal $k = 3$ (Figure 4).

Figure 4.

Elbow method applied to the stroke dataset. The inflection point around $k = 3$ suggests three latent clusters with distinct risk factor profiles.

Although these clusters did not align with stroke outcomes, they revealed subgroups with distinctive combinations of risk factors. One cluster grouped younger patients with elevated glucose levels, another older individuals with high BMI but no comorbidities. These profiles highlight that dichotomous classification can obscure clinically relevant differences. Unsupervised learning, while not predictive, offers a diagnostic lens: it surfaces ambiguities that supervised pipelines suppress. For pedagogy, this contrast is instructive – it shows that the same dataset can yield either silence (via classification) or plural signals (via clustering), depending on how feedback is conceptualized.