Manifold Modulation for Latent Space Steganography: A Capacity Bound and Distillation Demodulation Model for Teacher-Student Transmission Channels

In the paper's words

Large language models are increasingly deployed in teacher-student pipelines in which a student model is fine-tuned on text produced by a teacher, so any statistical signature the teacher imprints on its outputs is transferred to the student whether or not anyone intends it. No existing treatment gives a quantitative bound on covert bits per carrier token when the carrier task itself constrains the output distribution. We model the teacher's output distribution as a carrier subject to a per-token Kullback-Leibler (KL) modulation budget and derive a capacity bound C = D/ln 2 bits per token, where D is the budget in nats; at an illustrative budget of $D = 0.05$ nats this yields $0.072134$ bits per token, or $721.34$ bits ($90.17$ bytes) over a 10⁴-token carrier document.

The abstract, verbatim, and its opening sentence is the claim. Every line below is quoted from the text.

Numbers it reports

1. We model the teacher's output distribution as a carrier subject to a per-token Kullback-Leibler (KL) modulation budget and derive a capacity bound $C = D/\ln 2$ bits per token, where $D$ is the budget in nats; at an illustrative budget of $D = 0.05$ nats this yields $0.072134$ bits per token, or $721.34$ bits ($90.17$ bytes) over a $10^4$-token carrier document.

   from Abstract

2. A steganalysis sample-complexity bound showing that detection requires on the order of $\ln(1/\alpha)/D$ tokens, computed as $92.10$ tokens at $D = 0.05$ nats and $\alpha = 10^{-2}$ (Section 4).

   from 1. Introduction

3. A demodulation condition for distillation: a student recovers a behavior specification of $K$ bits from a carrier corpus of $M$ tokens when $M \geq K\ln 2 / D$, computed as $138{,}629$ tokens for $K = 10^4$ bits at the illustrative budget (Section 4).

   from 1. Introduction

4. The quantum steganography literature also demonstrates the failure mode that motivates our steganalysis bound: protocols built on BB84 quantum key distribution were shown to be vulnerable to a steganalysis attack that can detect the presence of the hidden transmission, which is why [4] develops modified protocols in which the party communicating steganographically operates in the reverse direction of the party sending the initial quantum states, camouflaged as part of the classical reconciliation step.

   from 2. Background and Related Work

5. GeMA combines deep generative modeling with classical efficiency-analysis concepts, learning latent manifold frontiers with a ProMan-VAE architecture, and its experiments probe settings where production technologies are heterogeneous, non-convex, or confounded with scale, using $n = 500$ decision-making units averaged over $30$ Monte Carlo replications [5].

   from 2. Background and Related Work

6. (i) The carrier task constrains the teacher's output distribution only through the budget $D$; we assume the carrier task's own entropy is large enough that a budget of $D \leq 0.2$ nats per token is feasible without truncating the carrier vocabulary.

   from 3.5 Assumptions

How it is done

1. No existing treatment gives a quantitative bound on covert bits per carrier token when the carrier task itself constrains the output distribution.

   from Abstract

2. If a quantitative capacity bound existed, a data provenance auditor could certify a corpus as carrying at most $B$ bits of upstream state, and a pipeline designer could deliberately encode desired behaviors into synthetic data at a known rate rather than by trial and error.

   from 1. Introduction

3. The quantum steganography literature also demonstrates the failure mode that motivates our steganalysis bound: protocols built on BB84 quantum key distribution were shown to be vulnerable to a steganalysis attack that can detect the presence of the hidden transmission, which is why [4] develops modified protocols in which the party communicating steganographically operates in the reverse direction of the party sending the initial quantum states, camouflaged as part of the classical reconciliation step.

   from 2. Background and Related Work

4. We define the carrier distribution $p^{(0)}_t(x) = p(x \mid x_{\lt t}, s_0)$ as the distribution the teacher would produce under a reference internal state $s_0$ that carries no covert information.

   from 3.1 Setting and notation

What would falsify it

1. First, its demonstration that a latent generative model can represent a structured frontier that classical estimators in observed space miss supports our claim that a low-dimensional covert manifold can be embedded in a high-dimensional output space without disturbing observable task statistics.

   from 2. Background and Related Work

2. Latent-space subdivision demonstrates that latent structure can be engineered to carry information orthogonal to the primary prediction task.

   from 2. Background and Related Work

3. The fluid-flow architecture of [6] splits the latent space so that external information can be passively injected into the prediction process: an obstacle unseen during training is injected by supplying an encoded density field with the obstacle region masked out, and the method predicts stable flow over $400$ time steps where the unsplit baseline fails and produces unphysical behavior [6].

   from 2. Background and Related Work

Where the claim stops

1. The result rests on the following assumptions, each with a distinct sensitivity: Per-token independence.

   from 6. Discussion

2. The capacity bound sums per-token mutual informations and assumes the modulation at each token is budgeted independently.

   from 6. Discussion

3. If the teacher modulates correlatedly across tokens (for example, encoding information in token-to-token transitions rather than marginal distributions), the per-token KL budget does not bound the joint information, and the true capacity can exceed $ND/\ln 2$.

   from 6. Discussion

Where to read

1. Abstract192 words · 1 min
2. 1. Introduction587 words · 3 min
3. 2. Background and Related Work917 words · 4 min
4. 3. Methods619 words · 3 min
5. 4. Analysis1,222 words · 5 min
6. 5. Results419 words · 2 min
7. 6. Discussion850 words · 4 min
8. 7. Practical Implications and Engineering Pathways784 words · 3 min
9. 8. Conclusion197 words · 1 min
10. Provenance and AI Involvement225 words · 1 min