Why Scientists May Stop Making Testable Predictions

In the paper's words

Scientific resource‑allocation systems—funding agencies, journals, tenure committees, and citation markets—reward novelty, narrative coherence, and impact while lacking a formal price for prospective falsification risk. This gap prevents quantitative assessment of whether such incentives systematically favor unfalsifiable research. We construct a minimal two‑strategy evolutionary‑game model in which agents choose between a Falsifiable (F) strategy (publish a testable prediction with accuracy p) and a Post‑hoc (P) strategy (publish a flexible narrative that avoids falsification).

The abstract, verbatim, and its opening sentence is the claim. Every line below is quoted from the text.

Numbers it reports

1. Quantity Formula & inputs Value Interpretation Accuracy threshold $p^{*}$ $(s+c)/(b+c)$, $s=1$, $c=1$, $b=2$ $0.6667$ Predictions must succeed > 66.7 % to be rational.

   from 5. Results

2. Using anchored illustrative values $b=2$, $c=1$, $s=1$, $p=0.6$, we derive the rationality threshold $$p^{*}= \frac{s+c}{b+c}= \frac{2}{3}\approx0.6667.$$ Because $p=0.6\lt p^{*}$, the P‑strategy yields a higher payoff ($\pi_P=1.0$ vs. $\pi_F=0.8$) and the replicator dynamics $\dot{x}=-0.2\,x(1-x)$ drive the falsifiable fraction to extinction.

   from Abstract

3. Embedding quality metrics such as the Triplet Generalization Ratio (TGR) are used to measure how well learned embeddings preserve narrative structure in social‑media posts, by sampling over 1,000 ground‑truth triplets, as reported in [1].

   from 2. Background and Related Work

4. Separating variables: $$ \frac{dx}{x(1-x)} = -0.2\,dt \;\Longrightarrow\; \ln\frac{x}{1-x}= \ln\frac{x_0}{1-x_0} -0.2\,t. $$ With $x_0=0.5$ and target $x=0.1$ (odds ratio reduced by factor $9$): $$ t = \frac{\ln 9}{0.2}\approx\frac{2.19722}{0.2}\approx10.99\text{ generations}. $$

   from 4.4 Replicator dynamics

5. Set $\pi_F=\pi_P$ and solve for $b$ (keeping $p=0.6$, $c=1$, $s=1$): $$ 0.6\,b - 0.4 = 1.0 \;\Longrightarrow\; b_{\text{flip}} = \frac{1.4}{0.6}= \frac{7}{3}\approx2.3333. $$ Equivalently, inverting the threshold: $$ b = \frac{s+c}{p} - c = \frac{2}{0.6} -1 = 3.\overline{3} -1 = 2.\overline{3}. $$ For any $b\gt b_{\text{flip}}$, the falsifiable strategy becomes individually optimal and, by the strict‑Nash argument, the all‑F state becomes the ESS.

   from 4.5 Flip point in the prediction reward

6. Keeping $b=2$, $c=1$, the new threshold $p^{*}_{\text{new}}= (2+1)/(2+1)=1.0$ and $\dot{x}= -1.2\,x(1-x)$, accelerating extinction of the falsifiable strategy sixfold.

   from 4.7 Reform scenarios

How it is done

1. We construct a minimal two‑strategy evolutionary‑game model in which agents choose between a Falsifiable (F) strategy (publish a testable prediction with accuracy $p$) and a Post‑hoc (P) strategy (publish a flexible narrative that avoids falsification).

   from Abstract

2. Derivation of the rationality threshold $p^{*}= (s+c)/(b+c)$ and proof that the Post‑hoc strategy is a strict Nash equilibrium, hence evolutionarily stable, whenever $p\lt p^{*}$ (Section 4).

   from 1. Introduction

3. We consider a large, well‑mixed population of agents (researchers).

   from 3.1 Model

4. Derive the individual‑rationality condition $\pi_F\gt \pi_P$ and solve for the accuracy threshold $p^{*}$.

   from 3.3 Analytic procedure

What would falsify it

1. The model predicts that increasing the effective prediction reward $b$ above the flip point raises the share of prospectively registered, falsifiable research .

   from 8.2 What would falsify the model

2. A pilot study that raises $b$ while holding $s$ and $c$ approximately constant, and observes no increase (or a decrease) in the fraction of registered predictions, would falsify the central claim.

   from 8.2 What would falsify the model

3. Scientific resource‑allocation systems—funding agencies, journals, tenure committees, and citation markets—reward novelty, narrative coherence, and impact while lacking a formal price for prospective falsification risk.

   from Abstract

Where the claim stops

1. Falsifiable practice is individually rational only when $p \gt p^{*}= (s+c)/(b+c)$; below this threshold the Post‑hoc strategy is a strict Nash equilibrium and therefore an evolutionarily stable strategy, with replicator dynamics driving falsifiable practice to extinction.

   from 9. Conclusion

2. Under the explicitly assumed parameters $b=2$, $c=1$, $s=1$, $p=0.6$, the threshold is $p^{*}=0.6667$, the payoff gap is $-0.2$, and the flip point is $b\approx2.3333$—a modest reward increase that reverses the equilibrium without altering reviewer judgment or researcher ability.

   from 9. Conclusion

3. Future work should incorporate network effects, heterogeneous accuracies, and multi‑stage decision processes, and should formalize the funder’s own optimization in the Stackelberg framework of [2].

   from 9. Conclusion

Where to read

1. Abstract170 words · 1 min
2. 1. Introduction250 words · 1 min
3. 2. Background and Related Work93 words · 1 min
4. 3. Methods231 words · 1 min
5. 4. Analysis369 words · 2 min
6. 5. Results163 words · 1 min
7. 6. Discussion1,022 words · 4 min
8. 6. Discussion249 words · 1 min
9. 7. Practical Implications and Engineering Pathways482 words · 2 min
10. 9. Conclusion178 words · 1 min
11. Version History54 words · 1 min
12. Provenance and AI Involvement225 words · 1 min