Metadata audit

I selected publicly documented training manifests that are commonly used in large language model (LLM) research: C4 and OpenWebText (The dataset is available in the TensorFlow Datasets catalogue: https://www.tensorflow.org/datasets/catalog/c4?hl=pl. The dataset for OpenWebText is available at: https://skylion007.github.io/OpenWebTextCorpus/. Finally, The Pile dataset is available at: https://pile.eleuther.ai/. All three datasets were used together to complete the research study.) as primary audit targets, with supplementary inspection of components of The Pile where manifests and provenance notes are available. These corpora were chosen because they are well-documented, widely cited in model training literature, and representative of the heterogeneous web-derived material that underpins many generative models.

The audit procedure is straightforward and fully reproducible. I ingested manifest files into a tabular environment (pandas) and normalised metadata fields (source URL, last_modified, dataset_name, checksum, size, and any deletion/flag fields). I defined a set of explicit criteria for identifying vestigial or potentially decayed records:

• • Soft-delete indicators: fields or flags that explicitly mark records as deleted/archived.

• • Accessibility failures: source URLs that return non-200 HTTPcodes, long redirect chains, or evidence of removal (404, 410).

• • Staleness: last_modified timestamps beyond a predefined threshold (I use 3 years as a working heuristic).

• • Provenance gaps: missing or inconsistent checksums, absent source attribution, or malformed metadata.

• • Duplicate and orphan signatures: identical checksums with divergent metadata or items cited by no other manifest entries.

• • Low metadata quality: records lacking language tags, author fields, or other curation signals used for filtering.

For each manifest, I scored records against these criteria and generated a ‘decay score’ (an additive index) to prioritise inspection. All processing was performed using open-source libraries (pandas, requests, validators) and documented in a public notebook to ensure reproducibility.

Case study: tracing decay through preprocessing

For a focused case study, I selected an openly documented model training pipeline that utilises portions of OpenWebText (hereafter referred to as the target pipeline). Using published preprocessing scripts and repository notes, I reconstructed the sequence of preprocessing steps commonly applied: retrieval → filtering/cleaning → deduplication → tokenisation → chunking → storage for training. Where available, I examined published filter lists, deduplication thresholds, and sampling heuristics.

The tracing exercise consisted of mapping audited manifest entries onto the pipeline stages to identify likely persistence points: for example, records flagged as ‘soft-deleted’ but included in archived snapshots; inaccessible source URLs that nonetheless appear in cached mirrors; and duplicate items that escaped deduplication owing to minor checksum mismatches. I recorded these mappings as annotated flow diagrams and tabular logs.

Prototype script – ‘data-compost’

To demonstrate a practical intervention, I developed a compact prototype script that implements the decay scoring, tags suspect records, emits structured logs, and exports a human-readable report (CSV/JSON). The script is intentionally minimalist, performing manifest ingestion, applying decay criteria, issuing severity tags, and writing an audit report that can be reviewed before any decision to exclude or reintegrate records. The prototype relies on Python (pandas, requests, loguru) and is runnable on a modest workstation.

To streamline the process of designing, iterating, and optimising the script, I employed Google AI Studio as a prototyping environment. Its code-assist features, rapid debugging capabilities, and capacity to simulate variations in pipeline logic allowed me to refine the workflow with unusual efficiency, while maintaining full transparency and control over the analytic logic. This hybrid approach – balancing human oversight with AI-supported optimisation – was especially valuable for a single-researcher project, ensuring both methodological rigour and feasibility.

From a reflexive standpoint, my use of Google AI Studio embodies the very interdisciplinarity that structures this research. On the one hand, the platform enabled me to enact principles of information management, such as lifecycle control and metadata accountability, by making the technical workflow auditable and reproducible. On the other hand, it highlighted the communicative and narrative stakes of intervention, as the design choices made in tagging, classifying, or discarding records echo broader debates about memory, omission, and storytelling in digital cultures. By situating myself at this junction – between technical design and interpretative critique – I position the prototype not merely as a tool for data hygiene but as an artefact that stages the tensions between governance, memory, and narrative framing.

Reflexivity and limitations

I approach this research as a scholar working at the intersection of information management and media/communication studies, privileging interpretative nuance alongside technical rigour. Consequently, the limitations of this desk-based, single-researcher approach must be explicitly acknowledged. While the metadata audits of open datasets are highly reproducible, this methodology cannot fully map or replicate the ‘black box’ proprietary preprocessing pipelines utilised by closed-source, industrial-scale AI developers. Furthermore, operating without the computational resources of a large laboratory, this study is restricted from performing end-to-end model retraining to empirically quantify the exact statistical degradation caused by specific ghost inputs. Therefore, the prototype prioritises transparency over scale; it functions as a heuristic device and a practical governance hook that is feasible for an individual researcher or small team to operationalise, rather than claiming to be a universal, industrial-grade solution.

Ghost inputs as active agents

My audit revealed a non-trivial number of records that met at least one decay criterion. Typical examples included forum posts archived with obsolete timestamps, blog posts whose source URLs now return 404, and archive snapshots that retained material explicitly flagged as deprecated in original repositories. Importantly, several such records were near-duplicates of higher-quality items; small textual differences (character encodings, trivial HTML artefacts) appear sufficient in practice to circumvent naive deduplication heuristics. To illustrate the delay criterion in practice, consider a specific scenario identified during the audit: a forum thread containing archaic, biased terminology was explicitly flagged as ‘soft-deleted’ by its host platform. However, because a mirrored version of the dataset retained an older timestamp without inheriting the updated deletion flag, the record bypassed the pipeline’s temporal staleness thresholds. The delay between the original act of deletion and the subsequent crawl of the unsynchronised mirror allowed this obsolete trace to persist as a valid training token years after its intended removal.

A parallel instance emerged within a digitised archive of underground journalism. In this case, a controversial article was formally retracted and scrubbed from the primary host server. Yet, a third-party Content Delivery Network (CDN) serving the site was configured with a prolonged time-to-live (TTL) cache policy. When an automated crawler scraped the domain during this temporal lag, it bypassed the updated primary database and instead ingested the obsolete CDN cache. The mismatch between the intentional act of erasure and the infrastructural refresh cycle effectively resurrected the retracted article, injecting it into the training corpus as an authoritative fact. Together, these scenarios demonstrate how minor temporal lags in distributed storage directly facilitate data decay, transforming infrastructural delays into epistemic vulnerabilities.

From an inferential perspective, these vestigial items operate as ghost inputs. They do not remain inert in the data ecology: when sampled into token streams, they alter co-occurrence frequencies, affect subtoken weightings, and thus nudge the statistical contours that generative models learn, as was observed in the study. In practical terms, I observed cases where decayed forum threads contained archaic or pejorative terminology. Where such entries are present in training material – even in small numbers – they can increase the likelihood of problematic token neighbours during decoding, thereby reinforcing latent bias (Caliskan et al. Reference Caliskan, Bryson and Narayanan2017). These observations align with documented cases in the literature where dataset artefacts correlate with biased model behaviours (Bolukbasi et al. Reference Bolukbasi, Chang, Zou, Saligrama and Kalai2016; Taniguchi et al. Reference Taniguchi, Sato and Shirakawa2018; Raji et al. Reference Raji, Gebru, Mitchell, Buolamwini, Lee and Denton2020; Bender et al. Reference Bender, Gebru, McMillan-Major and Shmitchell2021; Dodge et al. Reference Dodge, Sap, Marasović, Agnew, Ilharco, Groeneveld, Mitchell and Gardner2021), and they show how residual items can act as amplifiers even when not numerous.

The Data Decay Pathway: formal definition, schema

I define the Data Decay Pathway as the ordered set of loci and transitions through which vestigial digital artefacts migrate from archival or peripheral storage – a chaotic substrate that Bowker (Scientia Institute Rice University 2018) might term the ‘mnemonic deep’ – into active model substrates. Concretely, the Pathway is expressed formulaically as

$$ \mathbf{D}\mathbf{DP}=\mathbf{A}\to \mathbf{B}\to \mathbf{C}\to \mathbf{D}\to \mathbf{E}\to \mathbf{F}\to \mathbf{G}\to \mathbf{H}, $$

where:

• • A = Archival reservoir (cold storage, backups),

• • B = Manifest exposure (dataset listings, crawl indices),

• • C = Retrieval/caching (mirrors, CDNs),

• • D = Preprocessing (filters, deduplication, cleaning),

• • E = Tokenisation/chunking,

• • F = Training ingestion,

• • G = Model parameter landscape,

• • H = Deployment and downstream generation.

Linking information management with media, communication, and AI ethics for broader insights

From an information management perspective, the audit surfaces clear governance gaps. Many manifests lack standardised provenance fields; checksums are inconsistently recorded; and soft-delete semantics are poorly communicated (Oniszczuk and Makowska Reference Oniszczuk and Makowska2021). These gaps complicate lifecycle governance: without reliable metadata, it is difficult to assert whether a record should be retired, quarantined, or reviewed (Leipzig et al. Reference Leipzig, Nüst, Hoyt, Ram and Greenberg2021; Orzechowski et al. Reference Orzechowski, Opioła, Martínez, Ioannides, Panayiotou, Dutka, Słota and Kitowski2026). Accordingly, I propose minimal governance remedies – mandatory checksums, deletion flags in manifests, and a documented ‘decay score’ field – that are low friction yet high signal.

From a Communication and Media Studies standpoint, the presence of ghost inputs alters the texture of mediated recollection. Generative systems perform narrative stitching across heterogeneous fragments; when those fragments include decayed artefacts, the emergent storylines can be historically skewed, misattributed, or affectively misframed (Sturken Reference Sturken1997; Ernst Reference Ernst2002). A salient consequence is that mediated publics may come to remember versions of events that are composites of active and decayed inputs – an outcome with clear relevance for memory politics and public discourse.

Finally, in terms of AI Ethics and accountability, the Pathway exposes accountability gaps. Who is responsible when a model reproduces a harmful trope seeded by a decayed source? The answer is rarely straightforward: responsibility diffuses across crawlers, dataset curators, preprocessing teams, and model deployers. The Data Decay Pathway, therefore, becomes an accountability ledger: by instrumenting manifests and preprocessing stages with audit logs and decay tags, we create traceable points for responsibility and remediation. The ‘data-compost’ prototype exemplifies this approach by producing human-readable reports that can be used in audit trails, moderation review, and policy documentation.