Epistemic Legibility in AI-Assisted Science: Ignorance Auditing as a Governance Instrument for the Peer-Review and Evaluation Bottleneck

Epistemic Legibility in AI-Assisted Science: Ignorance Auditing as a Governance Instrument for the Peer-Review and Evaluation Bottleneck

Rowan Brad Quni-Gudzinas

QNFO, Amsterdam

(rowan@qnfo.org)

> Submitted to AI for Meta-Science workshop (NeurIPS 2026)

Abstract

As AI systems take on hypothesis generation, experiment planning, analysis, and writing, the scientific ecosystem faces a new bottleneck: not production, but evaluation, verification, and governance. This position paper contributes a documented case and an instrument to that problem. The case: a flagship paper produced inside an AI-assisted research pipeline was independently identified as AI-generated by structural, mathematical, and stylistic markers — while the forensic analyses that caught it simultaneously fabricated institutional and biographical claims of their own. The same pipeline that produced the unexamined artifact also produced, one day later, a systematic instrument for interrogating not-knowing: the Universal Ignorance Audit, a fifteen-question, five-phase method. We argue these are not coincidences but two faces of one governance problem: epistemic legibility — the degree to which a research pipeline can render legible what it knows, how it knows it, and what it does not know. We propose that AI-assisted research governance needs three legibility layers — provenance, ignorance, and auditor — and derive six transferable principles for AI reviewers, verification tools, and publication criteria: audit before asserting; disclose rather than conceal; verify provenance as a first-class gate; gate for generation-specific failure modes; invite adversarial validation; and audit the auditors.

1. Introduction

AI is transforming how scientific work is produced. Models now draft literature reviews, generate hypotheses, produce code, assemble complete papers, and increasingly execute elements of experimental workflows. The marginal cost of producing a scientific artifact is collapsing. The consequence is a bottleneck shift: the binding constraint on scientific progress is no longer production but evaluation, verification, and governance. Record submission numbers at major AI conferences and a growing reviewer crisis make this visible year after year.

This workshop's framing asks three questions: how to scale evaluation alongside production; how to detect and handle AI-generated content while responsibly incorporating AI assistance; and how to reorganize the scientific enterprise — what counts as a publication, how credit and incentives adapt.

Our position is that all three questions share a common substrate: epistemic legibility. An AI-assisted pipeline — or a peer-review system evaluating one — cannot govern what it cannot see. The failures that plague AI-assisted science (synthetic citations, scaffold-visible generation artifacts, unverifiable claims, fabrication) are legibility failures: features of the production process that are invisible to the evaluation process. We bring two things to this discussion that the typical position paper cannot: a documented case of an AI-generated paper caught inside the pipeline that produced it, and a field-tested instrument for making not-knowing legible.

The remainder of the paper: Section 2 documents the case — an AI-generated flagship paper and the forensic analyses that caught (and partially fabricated) it. Section 3 introduces the Universal Ignorance Audit as a governance instrument. Section 4 states our position: three legibility layers as the unit of analysis for AI-science governance. Section 5 derives six principles and maps them to concrete recommendations for the workshop's agenda — AI reviewers, verification tools, and publication criteria. Section 6 states limitations and open questions.

2. A Documented Case: The AI-Generated Flagship and Its Auditors

On 9 August 2026, two independent forensic analyses of a flagship paper published by QNFO (Quni-Gudzinas 2026b, DOI 10.5281/zenodo.21208346) concluded that the paper was AI-generated (documented in Quni-Gudzinas 2026c, DOI 10.5281/zenodo.21901983). The analyses converged on two classes of evidence.

Structural markers. Both analyses identified inlined meta-tags — bracketed labels such as [PHILOSOPHY], [speculative], [CHECK: 2027], Strength: [STRONG] | Status: [PENDING] — as signature outputs of prompt templates that instruct a model to label its own cognitive modes. They also identified synthetic citation anchors: citation keys with custom prefixes (@C5jpcubp0, @B1_shannon1948) that do not resolve to standard bibliographic entries. The paper's rigid scaffolding — pre-registered prediction registers, calibration registers, disconfirmation-condition tables — was identified as mimicking popular synthetic-evaluation frameworks designed to make AI text appear scientifically rigorous.

Mathematical and physical errors. The analyses identified a category error in the treatment of Landauer's bound: computing the bound at room temperature ($T = 300$ K) versus cryogenic temperature ($T = 10$ mK) in Planck units, and conflating thermodynamic erasure-energy floors with room-temperature operational coherence. The paper claimed that p-adic geometry and Bruhat–Tits trees could enable room-temperature quantum operation without proposing a physical hardware mechanism for suppressing thermal noise. The most sharply identified error was in the paper's decoder-energy treatment, where the decoder power was set inconsistently with the physical model.

The auditors' own failure. Crucially for this workshop: the forensic analyses that correctly identified structural markers of AI generation also fabricated institutional and biographical claims about the paper's provenance — claims that a proper ignorance audit would have flagged as scaffolds. The detection instrument was itself subject to the failure modes it purported to diagnose. AI-text detection is not an oracle; it is an epistemic instrument with its own blind spots, and it produced both a correct diagnosis and new fabrications.

Three lessons from the case:

1. Generation artifacts are legible when the pipeline does not hide them. The scaffold markers were visible precisely because the production process did not sanitize them. Detection worked against a naive pipeline — a pipeline with better post-processing would have defeated it.
2. Verification scales only if it is structured. The convergence of two independent analyses on the same structural and mathematical markers suggests that named, checkable categories of generation-specific failure (synthetic anchors, energy-budget errors, scaffold overload, self-referential metrics) are detectable at scale — but only if the evaluator is looking for them.
3. The auditor is inside the system. Forensic analysis of AI text is itself AI-adjacent cognition; its fabrications are the same failure mode unexamined. Governance that treats detectors as external oracles will inherit their blind spots.

3. The Instrument: The Universal Ignorance Audit

One day before the finding surfaced, the same organization developed, through iterative human–AI dialogue, the Universal Ignorance Audit (Quni-Gudzinas 2026a, DOI 10.5281/zenodo.21901984): a fifteen-question, five-phase method for systematically interrogating the structure of not-knowing in any domain.

The audit's starting point: not-knowing is not a void between islands of knowledge; it is an active structure with load-bearing members, blind corners, defended zones, and generative capacities. Its five phases move from (1) scaffolding — identifying the load-bearing assumptions the inquiry rests on; (2) map–territory hygiene — distinguishing the map from the territory; (3) wobble probing — locating felt anomalies; (4) power analysis — asking whose interests the current knowledge arrangement serves; and (5) recursive meta-audit — applying the audit to itself.

Three properties make it a governance instrument rather than a contemplative exercise:

• Content-independent. Its questions operate on the structure of any epistemic state — a paper, a review, a funding decision, an institutional belief. It does not require domain expertise to administer.
• Co-produced with AI, and self-applied to AI. The audit was built by a human and an AI assistant; its most valuable output came when it was turned on the AI's own frame, revealing an analytic, extractive, masculine slant and generating corrective sibling questions (relational ignorance, temporal patience, willful ignorance).
• Falsifiable administration. Its protocol requires explicit disconfirmation conditions, which converts vague epistemic unease into named, checkable structure.

The meta-audit's own finding is the key design fact for AI-science governance: the last unexamined scaffold is always the one doing the examining. This is the recursive structure that any governance layer for AI-assisted science must institutionalize, because it is the failure mode that detectors themselves exhibit (Section 2).

4. Position: Three Legibility Layers as the Unit of Governance

We propose that the peer-review and evaluation bottleneck is, at root, a legibility problem, and that AI-science governance should be organized around three legibility layers:

1. Provenance legibility — how was this produced? The minimal governance unit is not the artifact but its production record. Which parts were AI-generated, AI-assisted, or human-authored? Which prompts, models, and post-processing steps shaped the output? Current publication metadata does not capture this; the case in Section 2 shows that when provenance is invisible, evaluation degenerates into forensic archaeology. We advocate provenance as a required metadata field: a first-class gate, not an optional disclosure — with disclosure itself treated as a quality signal (disclosed AI involvement is legible; concealed involvement is an integrity violation).

2. Ignorance legibility — what does this not know, and does it know it does not know it? The most dangerous artifact in AI-accelerated science is not the wrong answer but the confident one that does not know it is wrong. The Universal Ignorance Audit operationalizes this as named, checkable structure: scaffolds, map–territory confusions, wobbles, protected ignorances. We advocate that evaluation protocols (AI reviewers, human reviewers, verification tools) explicitly probe for these categories — the audit's fifteen questions are a ready-made checklist.

3. Auditor legibility — who audits the auditors? Every verification layer is a map and must itself be audited, or the error compounds. The forensic analyses in Section 2 fabricated claims while detecting fabrication; a reviewer-crisis response built on AI reviewers will inherit the same structure unless the review process itself carries an audit trail. We advocate that every AI-assisted review artifact carry the same provenance and ignorance metadata as the artifacts it reviews.

5. Six Principles and Workshop Recommendations

From the case (Section 2) and the instrument (Section 3), we derive six transferable principles for AI-assisted research pipelines — and map them to concrete agenda items for this workshop:

The workshop's stated outcomes — a position paper with recommendations, guidelines for ML communities on the peer-review crisis, and a lasting working group — are, in our view, the right vehicles for institutionalizing these six principles. The empirical base for them exists: this case is one documented instance of what happens when a production pipeline and its evaluation layer both lack legibility.

6. Limitations and Open Questions

We state the limits of our contribution candidly.

• Single-case evidence. The case is one organization's documented experience. It is existence-proof that the described failure modes occur and are detectable, not a prevalence estimate. Prevalence research is an urgent open question for the meta-science community.
• Self-referential risk. The audit is offered by the same kind of pipeline it governs. This is deliberate (Principle 6) and the reason the audit's recursive meta-question is part of the protocol — but readers should treat the instrument as itself unaudited until independent evaluation exists.
• Instrumentation cost. The audit's administration is not free; scaling it to high-throughput review requires tooling that does not yet exist. We see this as a design target for the workshop's working group.
• Adversarial pipelines. Detection and auditing work against cooperative or naive pipelines. An adversary with better post-processing can remove scaffold markers. Legibility governance raises the cost of concealment; it does not eliminate it. Open question: what governance structures (incentives, liability, audit) are robust to adversarial production?

7. Conclusion

The bottleneck in AI-accelerated science is evaluation, verification, and governance. Our position is that this bottleneck is a legibility problem with a recursive structure: pipelines that cannot see their own scaffolds, detectors that fabricate while detecting, and review systems that scale by inheriting the blind spots of their instruments. The Universal Ignorance Audit is a field-tested instrument for making not-knowing legible; the documented case shows both the cost of its absence and the danger of naive detection. We invite the workshop to organize its working group around the three legibility layers — provenance, ignorance, auditor — and to adopt the six principles above as the skeleton of its community guidelines. The last unexamined scaffold is always the one doing the examining; the only governance that works is the one that audits itself.

References

Quni-Gudzinas, R. B. (2026a). The Universal Ignorance Audit: A Fifteen-Question Method for Systematic Inquiry into the Structure of Not-Knowing. Zenodo, v0.3. DOI: 10.5281/zenodo.21901984.

Quni-Gudzinas, R. B. (2026b). The Ultrametric Foundation: A Unified Thesis on Number, Time, Knowledge, and Computation. Zenodo. DOI: 10.5281/zenodo.21208346.

Quni-Gudzinas, R. B. (2026c). Knowing What We Do Not Know: Ignorance Auditing, AI-Generation Detection, and the Epistemic Lessons of an AI-Assisted Research Pipeline. Zenodo, v0.3. DOI: 10.5281/zenodo.21901983.

Firestein, S. (2012). Ignorance: How It Drives Science. Oxford University Press.

Landauer, R. (1961). Irreversibility and heat generation in the computing process. IBM Journal of Research and Development, 5(3), 183–191.

Merton, R. K. (1987). Three fragments from a sociologist's notebooks: Establishing the phenomenon, specified ignorance, and strategic research materials. Annual Review of Sociology, 13, 1–29.

Proctor, R. N., & Schiebinger, L. (Eds.) (2008). Agnotology: The Making and Unmaking of Ignorance. Stanford University Press.

Tetlock, P. E. (2005). Expert Political Judgment: How Good Is It? How Can We Know? Princeton University Press.

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.

Whitcomb, D., Battaly, H., Baehr, J., & Howard-Snyder, D. (2017). Intellectual humility: Owning our limitations. Philosophy and Phenomenological Research, 94(3), 509–539.