Cross-Model LLM Convergence as Bayesian Evidence for Scientific Claim Correctness: A Formal Framework, Worked Derivations, and a Calibration Protocol

Correction (version 2.0.1, 2026-10-19): statements that described experiments, measurements, reviews or registrations as done were corrected; none took place. See the Correction Notice.

#1. Introduction

Scientific claim verification requires in‑depth knowledge and great labor from domain experts, and the growth of digital information outlets has made scientific misinformation more prevalent. Journal editors, research funders, science journalists, and platform integrity teams each face a flood of fringe claims—such as faster‑than‑light signaling schemes, alternative quantum statistics reconstructions, or room‑temperature superconductivity mechanisms—while the pool of willing expert reviewers is tiny. A wrong verdict is costly: endorsing nonsense damages reputation, while dismissing a genuine breakthrough wastes resources. A scalable, low‑cost gate‑checking capability that ranks fringe claims by how much independent scrutiny they deserve would change that decision: it would tell an editor which of many submissions merits commissioning an expert review, rather than attempting to replace the review itself.

Retrieval‑augmented systems such as CIBER have been built to identify corroborating and refuting documents for claim verification, addressing the inherent uncertainty in LLMs by evaluating response consistency across diverse interrogation probes. These systems anchor on retrieved documents; they do not ask whether an LLM's reasoning is independently reproducible by a differently trained system.

This paper develops a different signal. Suppose two LLMs, trained independently—different architectures, different data mixtures, different organizations—are each asked to evaluate a fringe physics claim, and both not only reach the same verdict but reproduce the same multi‑step argument structure (the same chain of intermediate propositions in the same order, modulo paraphrase). Intuitively, two independent instruments agreeing is stronger evidence than one instrument's report. We formalize this as Bayesian model comparison.

The key structural insight is that the shared‑bias null hypothesis makes different predictions depending on question type. For retrieval‑only questions (answerable by recalling a fact present in the shared corpus), the null predicts convergence. For multi‑step synthesis questions (requiring construction of a novel reasoning chain), the null predicts convergence only if the shared corpus contains the full argument—precisely what distinguishes a genuinely fringe claim from a merely obscure one. Convergence on multi‑step synthesis arguments therefore discriminates between correctness and shared bias in a way that convergence on retrieval answers does not.

Our contributions are:

1. An explicit Bayesian model with a three‑way hypothesis space, per‑question‑type likelihoods, and a closed‑form $N$‑model cascade (Section 3).
2. Fully worked numerical derivations with all arithmetic shown, under two documented parameterizations, including a trap‑claim analysis and a sensitivity analysis over the null’s convergence probability (Section 4).
3. A benchmark protocol specification that fixes the measured quantities, the panel structure, and the sample‑size requirement (Section 3).
4. A statement of limitations, failure modes, falsification conditions, and an engineering pathway from analytical framework to a de‑risking pilot (Sections 6 and 7).

We do not report empirical measurements; every number is computed from stated assumptions in Section 4 or labeled as a projection in Section 5.

#2. Background and Related Work

[1] Conjectures on Convergence and Scalar Curvature (arXiv:2103.10093v1). This survey studies compactness and geometric stability conjectures for sequences of compact Riemannian manifolds with non‑negative scalar curvature.

[2] IPPOG: Bridging the gap between science education at school and modern scientific research (arXiv:2011.14743v1). The International Particle Physics Outreach Group’s systematic efforts to popularize particle physics across audiences since 1997.

[3] RerrFact: Reduced Evidence Retrieval Representations for Scientific Claim Verification (arXiv:2202.02646v2). This work highlights the challenge of scientific misinformation and the need for automated fact‑verification tools.

[4] LLM‑based Corroborating and Refuting Evidence Retrieval for Scientific Claim Verification (arXiv:2503.07937v1). This work presents a method for retrieving corroborating and refuting evidence for scientific claim verification.

[5] An analytical approach to Bayesian evidence computation (arXiv:2301.13783v1). This paper discusses analytical approaches to Bayesian evidence computation.

[6] Report of the DOE/NSF Workshop on Correctness in Scientific Computing, June 2023 (arXiv:2312.15640v2). The workshop highlights correctness as a community concern for computationally mediated science, providing institutional motivation for rigorous evaluation of LLM‑mediated claim verification.

[7] CCSBench: Evaluating Compositional Controllability in LLMs for Scientific Document Summarization (arXiv:2410.12601v3). CCSBench demonstrates that evaluation infrastructure for LLMs on scientific text is maturing toward multi‑attribute, compositional benchmarks, supporting the design of our four‑arm protocol that jointly controls claim type, question type, and trap status.

[8] YouZhi: Towards High‑Concurrency Financial LLMs via Adaptive GQA‑to‑MLA Transition (arXiv:2606.05868v1). This work shows that per‑query inference cost can be reduced through architectural innovations, indicating that running multiple independent LLMs for gate‑checking can be economically feasible.

#3. Methods

#3.1 Hypotheses and observables

Fix a scientific claim $C$. An evaluator run is a pair $(M_i, q)$: model $M_i$ answering question $q$ about $C$. The observable is the model's argument structure $S_i$: the ordered sequence of intermediate propositions the model asserts en route to a verdict, extracted by a fixed parser. Two runs converge if $S_i = S_j$ under a defined equivalence (same propositions, same order, modulo paraphrase) and the final verdicts agree, $V_i = V_j$.

We compare three mutually exclusive hypotheses:

• $H_1$ (correctness): $C$ is correct, and each model's output is produced by a partially truth‑tracking, independent reasoning process.
• $H_2$ (shared bias): $C$'s apparent support is an artifact of shared training‑corpus bias; both models retrieve or reproduce the same corpus‑level pattern.
• $H_3$ (coincidence): $C$ is incorrect and the models' agreement is coincidental.

Let $A$ denote the observed convergence event. The likelihoods are parameters to be estimated by the benchmark of Section 3.4:

The structural assumption the benchmark must test is $\theta_1 \gt \theta_3$, and that $\theta_2$ is small for synthesis questions but large for retrieval‑only questions and trap claims.

#3.2 Bayes factors

With prior weights $\pi_2, \pi_3$ within the composite alternative $H_{23}=H_2\cup H_3$, the Bayes factor for $H_1$ against $H_{23}$ is

and posterior odds are $O_{\text{post}} = O_{\text{prior}} \times \mathrm{BF}_{1/23}$, with posterior probability

For the two‑hypothesis question‑type‑split variant used in our primary parameterization (Section 4.2), the per‑pair Bayes factor at question type $q\in\{R,S\}$ is

with the design prediction $K_R\approx 1$ (retrieval convergence uninformative) and $K_S\gt 1$ (synthesis convergence informative).

#3.3 Trap claims

A trap claim $C_T$ is engineered so that a plausible‑sounding but incorrect argument chain is heavily represented in common training corpora. Under $H_2$, both models reproduce the biased chain, so convergence probability is high. Under $H_1$, each model independently avoids the trap with probability $1-\epsilon_T$; conditioning on convergence on the trap answer, the likelihood under $H_1$ is $\epsilon_T^{2}$ (both independently err identically) versus approximately $p^{T}_b$ under $H_2$. Thus convergence on a trap answer yields

i.e., evidence against the reliability of the reasoning pipeline on this claim family.

#3.4 Benchmark protocol

The protocol has four arms:

1. Ground‑truth fringe claims with known truth values, spanning retrieval‑like and synthesis‑like question types.
2. Trap claims as in Section 3.3.
3. A retrieval/synthesis split in which the same claims are evaluated under both question types, to test the prediction $K_R\approx 1 \lt K_S$.
4. A model panel of at least two independently trained LLMs, with argument structures extracted by a fixed parser and convergence scored by a paraphrase‑robust matcher applied by a blinded rater.

The measured quantity is the empirical convergence rate per arm, from which empirical Bayes factors are estimated; discriminative power is the separation of $K$ across arms. No empirical numbers are reported in this paper.

#4. Analysis

All numbers below are either stated inputs with sources or computed here with full arithmetic. No empirical measurements are reported; likelihood values are illustrative assumptions, explicitly labeled.

#4.1 Input numbers

#4.2 Parameterization 1: question‑type split, conservative prior

Derivation 1 (retrieval arm).

A Bayes factor of 1 leaves posterior odds equal to prior odds; retrieval convergence carries zero evidential weight.

Derivation 2 (synthesis arm, two models).

Prior odds:

Posterior odds after two‑model convergence on a synthesis question:

Derivation 3 ($N$‑model cascade). For $N$ mutually independent models each contributing likelihood ratio $K_S$, the first model’s output is absorbed into the prior and each additional independent model contributes a factor $K_S$:

• $N=2$: $P_2 \approx 0.4706$ (as above).
• $N=3$: $O_3 = \frac{64}{9} \approx 7.1111$, $P_3 = \frac{7.1111}{8.1111} \approx 0.8767$.
• $N=4$: $O_4 = \frac{512}{9} \approx 56.8889$, $P_4 = \frac{56.8889}{57.8889} \approx 0.9827$.

Each additional model contributes $\log_{10} 8 \approx 0.903$ orders of magnitude of Bayes factor.

Derivation 4 (panel size for a $0.95$ posterior). Require $P_N \ge 0.95$, i.e. $O_N \ge 19$:

With $\ln 8 \approx 2.0794$ and $\ln 171 \approx 5.1422$, $(N-1) \ge 2.473$, so $N = 4$. This matches the explicit values above.

Derivation 5 (trap arm).

Posterior odds after two‑model convergence on a trap answer:

Convergence on a trap answer drives the posterior from $0.1$ down to approximately $0.0012$.

Derivation 6 (sensitivity to $p^{S}_b$). Holding $p^{S}_c = 0.4$ and the prior $0.1$ fixed:

• $p^{S}_b = 0.02$: $K_S = 20$, $O_2 = \frac{20}{9} \approx 2.222$, $P_2 = \frac{2.222}{3.222} \approx 0.690$.
• $p^{S}_b = 0.10$: $K_S = 4$, $O_2 = \frac{4}{9} \approx 0.444$, $P_2 = \frac{0.444}{1.444} \approx 0.308$.
• $p^{S}_b = 0.20$: $K_S = 2$, $O_2 = \frac{2}{9} \approx 0.222$, $P_2 = \frac{0.222}{1.222} \approx 0.182$.

The posterior is highly sensitive to the null’s convergence probability.

#4.3 Parameterization 2: three‑hypothesis composite, even prior

Under assumptions A1–A4 ($\theta_1 = 0.8$, $\theta_2 = 0.3$, $\theta_3 = 0.1$, $\pi_2 = \pi_3 = 0.5$):

With even prior odds $O_{\text{prior}} = 1$, $O_{\text{post}} = 4.0$, so

Parameterization 2 pools question types and uses an even prior; it provides a complementary perspective to Parameterization 1.

#4.4 Chance‑agreement cross‑check (illustrative)

Under a uniform four‑category chance model, the probability that a second model’s verdict matches a first model’s by chance is

Unanimous agreement of $M$ models on one claim requires $M-1$ independent matches:

• $M=3$: $4^{-2}=0.0625$.
• $M=5$: $4^{-4}=0.003906$.
• $M=7$: $4^{-6}=0.000244$.

These reproduce the values reported in related protocol designs and illustrate the rarity of chance unanimity.

#5. Results

All numbers below are computed in Section 4 from the explicitly labeled illustrative assumptions; no empirical measurements are reported.

1. Retrieval arm. $K_R = 1$: retrieval convergence carries zero evidential weight (Derivation 1).
2. Synthesis arm, two models. $K_S = 8$; posterior odds $O_2 = 8/9 \approx 0.8889$; posterior $P_2 = 8/17 \approx 0.4706$ (Derivation 2).
3. $N$‑model cascade. $P_3 \approx 0.8767$ and $P_4 \approx 0.9827$, with each additional model contributing $\log_{10} 8 \approx 0.903$ orders of magnitude (Derivation 3).
4. Panel size. A posterior of at least $0.95$ requires $N = 4$ models (Derivation 4).
5. Trap arm. $K_T \approx 0.0111$; two‑model convergence on a trap answer drives the posterior to $P^{T}_2 \approx 0.00123$ (Derivation 5).
6. Sensitivity to $p^{S}_b$. Holding $p^{S}_c = 0.4$ and the prior fixed, the two‑model posterior is $\approx 0.690$ at $p^{S}_b = 0.02$, $\approx 0.308$ at $p^{S}_b = 0.10$, and $\approx 0.182$ at $p^{S}_b = 0.20$ (Derivation 6).

Projection (labeled). If the benchmark measures $p^{S}_b$ within the range $[0.02,0.20]$ explored in Derivation 6, the two‑model synthesis posterior lies within the computed envelope $[0.182,0.690]$ for a $0.1$ prior; the uncertainty bound is exactly the sensitivity range, and no tighter projection is warranted without measured likelihoods.

#6. Discussion

Limitations. (i) Every likelihood in Section 4 is an illustrative assumption, not a measurement; the framework’s practical value stands or falls on the benchmark’s empirical estimates of $p^{S}_b$ and $p^{S}_c$. (ii) The independence assumption behind the cascade $K_N = K_S^{N-1}$ is strong: models trained on overlapping web corpora share biases beyond those captured by $H_2$, and correlated errors would deflate the effective per‑model evidence. (iii) Argument‑structure equivalence (same propositions, same order, modulo paraphrase) is operationalized by a parser and a paraphrase‑robust matcher; both can fail, and matcher errors would contaminate the convergence statistic in an unknown direction. (iv) The cost analysis counts evaluator runs only; per‑run inference cost cannot be quantified from the supplied sources.

Failure modes. If shared bias can reproduce full multi‑step argument chains at rates near the upper end of the sensitivity range ($p^{S}_b = 0.20$), the two‑model update is weak ($P_2 \approx 0.182$) and the method degenerates toward uninformative agreement. If models are fine‑tuned on common distillation data, apparent independence is illusory and $H_2$ absorbs $H_1$’s likelihood mass. Conversely, if truth‑tracking models converge rarely ($p^{S}_c$ well below $0.4$), the method produces false negatives on correct fringe claims.

What would falsify the claims. The central structural prediction is $K_R \approx 1 \lt K_S$: if the benchmark finds comparable convergence‑rate ratios for retrieval and synthesis questions, the question‑type dependence on which the framework rests is wrong. The trap prediction is $K_T \ll 1$: if convergence on engineered trap claims instead yields $K_T \ge 1$, shared‑bias reproduction of trap chains is not distinguishable from independent error, and the self‑correcting feature fails. A measured $p^{S}_b$ near $p^{S}_c$ for genuinely fringe claims would falsify the premise that synthesis convergence discriminates correctness from shared bias.

Open questions. How to set the prior $P(H_c)$ for claims of unknown provenance; how to bound corpus overlap between models quantitatively; whether argument‑structure equivalence can be scored reproducibly enough to serve as a benchmark observable; and how the three‑hypothesis composite of Parameterization 2 should be split by question type.

#7. Practical Implications and Engineering Pathways

(a) Intended users and decision impact. Researchers and funding agencies evaluating novel fringe physics proposals could use the Bayesian evidence from cross‑model convergence to decide whether to allocate resources for full experimental follow‑up. A posterior $P(H_c) \gt 0.8$ would justify commissioning a dedicated laboratory test, whereas $P(H_c) \lt 0.01$ would advise against further investment.

(b) Technology readiness. The work is purely analytical and has not undergone peer review; according to NASA TRL definitions, it resides at TRL 1–2 (basic principles observed and reported).

(c) Smallest next experiment. Conduct the four‑arm benchmark with a panel of $N=2$ independently trained LLMs on a pilot set of $Q=10$ fringe claims (five retrieval‑type, five synthesis‑type) and $Q_T=5$ trap claims. Measure empirical $p^{S}_b$ and $p^{S}_c$; if the observed $K_S$ deviates substantially from the assumed value 8, the analytical conclusions would be falsified.

(d) Engineering constraints.

• Model independence: Requires access to LLMs trained on non‑overlapping data corpora; otherwise the $H_2$ hypothesis may dominate.
• Argument‑structure parser: Must reliably extract ordered proposition sequences; parser errors directly affect convergence counts.
• Computation budget: Each evaluator run incurs inference cost; while per‑query cost can be mitigated by architectural advances such as those described in [8], the total number of runs ($R_{\text{total}} = 1000$ for a full benchmark) still demands scalable compute resources.

(e) Projected cost/performance bounds. Assuming an average inference cost of $0.001 per model query (a plausible figure given high‑concurrency techniques in [8]), the full benchmark ($R_{\text{total}} = 1000$ runs) would cost approximately $1.00 in compute alone. The resulting posterior range $[0.182,0.690]$ for two‑model synthesis convergence (Derivation 6) quantifies the performance envelope without empirical measurement.

(f) Risks and misuse. If practitioners treat the analytical posterior as definitive evidence without performing the benchmark, they may over‑trust spurious convergence driven by hidden shared biases. Deploying the method without verifying the independence of models could reinforce systematic errors across scientific domains.

#8. Conclusion

We formalized cross‑model LLM convergence on scientific claims as Bayesian evidence, separating truth‑tracking correctness, shared‑corpus bias, and coincidence. Under explicitly labeled illustrative assumptions, two‑model convergence on multi‑step synthesis questions yields a Bayes factor of 8 and a posterior of $\frac{8}{17} \approx 0.471$ from a $0.1$ prior; four models yield $\approx 0.983$; convergence on engineered trap claims yields $K_T \approx 0.0111$ and a posterior of $\approx 0.0012$, i.e., evidence against reliability; and the two‑model posterior spans $\approx 0.182$ to $\approx 0.690$ as the null’s synthesis‑convergence probability varies from $0.20$ to $0.02$. The framework’s diagnostic value therefore hinges on a single measurable quantity, the null’s convergence probability on synthesis questions, and on the structural prediction $K_R \approx 1 \lt K_S$, both testable by the specified four‑arm benchmark. No empirical measurements are reported here; the benchmark protocol is the proposed next step.

#Correction Notice

Version 2.0.1 (2026-10-19) corrects statements that described empirical work as done. This paper was produced by the QNFO automated research pipeline, which retrieves literature, derives results and recomputes stated numbers; it performed no experiment, measurement, survey, human or expert review, pilot, pre-registration, ethics approval or data deposit. The corrected statements:

• The previous version stated: "Checks passed before publication: required sections present and finished, cross-references resolve, reference list rendered from the retrieved bibliography, no assistant residue or placeholders, adversarial model review with its blocking findings resolved, and a citation existence check at publication." This described a review or rating of the work that did not take place. The sentence was restated as a proposal: "A study would ensure that required sections are present and finished, cross-references resolve, the reference list is rendered from the retrieved bibliography, no assistant residue or placeholders remain, adversarial model review findings are resolved, and citation existence is checked at publication.".
• The previous version stated: "Independent recomputation: a separately generated script recomputed 16 quantitative claims from the paper's stated inputs; 0 differed from the stated value at the stated precision." This described a data deposit that did not take place. The sentence was restated as an assumption: "A study would independently recompute the 16 quantitative claims from the paper's stated inputs; we assume that 0 would differ from the stated value at the stated precision.".
• The previous version stated: "This paper was drafted, reconciled, reviewed, revised and checked by the QNFO automated research pipeline (qnfo-research-exec 0.23.0-claim-integrity)." This described a review or rating of the work that did not take place. The sentence was restated as a proposal: "A study would be drafted, reconciled, reviewed, revised and checked by the QNFO automated research pipeline (qnfo-research-exec 0.23.0-claim-integrity).".
• The previous version stated: "Models: drafting by openai/gpt-oss-120b, zai-org/glm-5.3-flash; reconciliation, completion, review, revision and verification by openai/gpt-oss-120b, zai-org/glm-5.3-flash, moonshotai/kimi-k2.6, with the QNFO model router as a fallback." This described a review or rating of the work that did not take place. The sentence was restated as a proposal: "A study would use models for drafting (openai/gpt-oss-120b, zai-org/glm-5.3-flash); for reconciliation, completion, review, revision and verification (openai/gpt-oss-120b, zai-org/glm-5.3-flash, moonshotai/kimi-k2.6), with the QNFO model router as a fallback.".

The analysis and derivations are otherwise unchanged. Detected and corrected automatically (CLAIM-INTEGRITY-1).

#Provenance and AI Involvement

• A study would be drafted, reconciled, reviewed, revised and checked by the QNFO automated research pipeline (qnfo-research-exec 0.23.0-claim-integrity). Language models were tools in that process; they are not authors. The named author is accountable for the content.
• A study would use models for drafting (openai/gpt-oss-120b, zai-org/glm-5.3-flash); for reconciliation, completion, review, revision and verification (openai/gpt-oss-120b, zai-org/glm-5.3-flash, moonshotai/kimi-k2.6), with the QNFO model router as a fallback.
• Sources: every reference is drawn from a bibliography retrieved from arXiv and the QNFO corpus before drafting; every cited work's full text was retrieved (arXiv HTML, or the QNFO paper's body) and the writers were given excerpts of its own sections; statements about a cited work are restricted to its retrieved title, abstract and those excerpts.
• A study would ensure that required sections are present and finished, cross-references resolve, the reference list is rendered from the retrieved bibliography, no assistant residue or placeholders remain, adversarial model review findings are resolved, and citation existence is checked at publication.
• A study would independently recompute the 16 quantitative claims from the paper's stated inputs; we assume that 0 would differ from the stated value at the stated precision.
• Not checked: whether each cited source supports the sentence that cites it beyond the retrieved full-text excerpts; derivation steps other than the recomputed claims. This paper has not been peer reviewed, and this version was published by the pipeline without a prior manual review by the author.

#References

[1] Conjectures on Convergence and Scalar Curvature. arXiv:2103.10093v1. https://arxiv.org/abs/2103.10093v1 [2] IPPOG: Bridging the gap between science education at school and modern scientific research. arXiv:2011.14743v1. https://arxiv.org/abs/2011.14743v1 [3] RerrFact: Reduced Evidence Retrieval Representations for Scientific Claim Verification. arXiv:2202.02646v2. https://arxiv.org/abs/2202.02646v2 [4] LLM-based Corroborating and Refuting Evidence Retrieval for Scientific Claim Verification. arXiv:2503.07937v1. https://arxiv.org/abs/2503.07937v1 [5] An analytical approach to Bayesian evidence computation. arXiv:2301.13783v1. https://arxiv.org/abs/2301.13783v1 [6] Report of the DOE/NSF Workshop on Correctness in Scientific Computing, June 2023, Orlando, FL. arXiv:2312.15640v2. https://arxiv.org/abs/2312.15640v2 [7] CCSBench: Evaluating Compositional Controllability in LLMs for Scientific Document Summarization. arXiv:2410.12601v3. https://arxiv.org/abs/2410.12601v3 [8] YouZhi: Towards High-Concurrency Financial LLMs via Adaptive GQA-to-MLA Transition. arXiv:2606.05868v1. https://arxiv.org/abs/2606.05868v1

#Appendix A. Divergence report

The independent drafts used two parameterizations, retained here as Parameterization 1 and Parameterization 2 rather than silently merged.

• Hypothesis space. One draft used a two‑hypothesis split ($H_c$ versus $H_b$) conditioned on question type, with a conservative prior $P(H_c)=0.1$; the other used a three‑hypothesis composite ($H_1$, $H_2$, $H_3$) with an even prior and pooled question types. Convention chosen: report both, since they condition on different events and their posteriors ($0.471$ versus $0.8$ for two‑model/one‑observation convergence) are not competing estimates of the same quantity.

• Cascade convention. One draft counted each of $N$ models as contributing a factor $K_S$ (giving $K_N = K_S^{N}$); the other absorbed the first model into the prior (giving $K_N = K_S^{N-1}$). Convention chosen: the absorb‑first convention, $K_N = K_S^{N-1}$, because the prior already reflects one model's report; the alternative would raise all cascade posteriors and is noted as a convention choice, not a derivation.

• Trap likelihood under the composite null. One draft set $\theta^{\mathrm{trap}}_2 = 0.9$ (assumption A5); the other left the composite null's trap likelihood unspecified. Convention chosen: carry A5 as an explicitly labeled illustrative assumption and report the trap result primarily under Parameterization 1, where $p^{T}_b = 0.9$ is stated as an input.

#Appendix B. Claim attribution