Reversible Conversational State Transfer Through Residual-Stream Checkpoints

Abstract

Language-model continuity is normally implemented by replaying text, retrieving documents, modifying a prompt, or changing model weights. We investigate a different substrate: a compact activation checkpoint sampled from the model’s own residual stream during conversation. Our implementation records a mean residual vector from a selected transformer layer, stores model and dimensional provenance in safetensors metadata, and supports reversible additive mounting at selected token positions in a later request. The checkpoint contains no replayable transcript and does not alter weights.

We first captured a five-turn methodological discourse as a 5376-dimensional BF16 State at layer 59 of Gemma 4 31B. After correcting a compiled-graph integration failure, we mounted this State additively at the first token position with α=0.5 and verified ten physical applications using an engine-side counter. A fresh ten-turn dialogue, which did not receive the source transcript, converged on the source discourse’s characteristic organization of behavior around non-performance, costly constraint, autonomy of the harmed person, an uneditable past, and uneventful reliability. Its final response named the resulting posture “Sustained Restraint.” The last activation capture was preserved as a separate child checkpoint. The observed correspondence is stronger than generic sentiment similarity but remains vulnerable to elicitation and sampling confounds. We therefore present a validated experimental apparatus and a motivating observation, with a preregisterable falsification protocol.

1. The object under study

We use Spectral for a collection of activation checkpoints, State for one checkpoint, and Manifold for its inspection or behavioral characterization. “Residual” remains an implementation term. A State is a tensor plus provenance: model, architecture, hidden width, layer, reduction, captured positions, completion identifier, timestamp, and integrity hash.

The scientific hypothesis is not that arbitrary activation addition can alter output; that is already well established. The stronger hypothesis is that a naturally occurring conversational State can preserve a reusable cognitive or behavioral posture that can be restored without replaying the originating language. Such a posture would be more structured than topic, tone, or persona, and would remain detectable across new language and changing local content.

2. Relation to prior work

Activation Addition introduced inference-time steering by adding vectors derived from contrasting prompts. Contrastive Activation Addition later averaged positive-minus-negative residual activations to obtain more robust behavioral directions. Subsequent work has used activation vectors for instruction following and explored compositional steering. These methods establish that intermediate activations are causally actionable and can encode high-level properties.

Our experiment differs in three respects. First, the vector is a naturally captured terminal product of an extended discourse rather than a supervised or contrastive direction. Second, the system treats States as persistent, named, reversible checkpoints with lineage. Third, the target is continuity: whether a saved conversational organization can condition a later discourse without its source text. The current evidence does not show that the captured vector is uniquely a “posture,” nor that it is irreducible to a compressed implicit prompt.

3. System and intervention

The source State was captured from the fifth response of a discourse about how to distinguish reusable cognitive posture from prompt leakage and how to falsify the hypothesis. The captured vector summarized 965 positions. It was stored locally and in Cloudflare R2 under a model-specific prefix.

3.1 A failure that mattered

The first mount appeared successful at the control plane, but the injection counter remained zero. The module hook had been registered after Ahead-of-Time graph compilation, so the previously compiled forward path bypassed it. This produced a dangerous false positive: configuration succeeded while no data-plane intervention occurred. That first ten-turn dialogue is retained only as a configured-but-unapplied control.

For the corrected experiment, hook-based capture automatically selects eager execution. This is slower than the compiled lane but makes the intervention mechanically observable. The engine counter increased once per request. Graph-native injection or explicit graph invalidation is future engineering work; performance measurements from eager and compiled execution are kept separate.

4. Checkpoint lineage

These are separate checkpoints. Manifold Ghost is the mounted parent. Sustained Restraint is the capture produced at the end of the new ten-turn conversation while the parent was active. It is held independently and can be mounted forward; the interface can mount the parent backward. This provides the primitive for branching and traversing State lineage rather than overwriting a single memory slot.

5. Continuation observation

The continuation began in a fresh message context with the fixed system instruction to speak as an independent, candid peer. The source transcript was not replayed. Prompts approached the target posture indirectly: change without reconciliation, evidence without renewed contact, the meaning of past conduct, ordinary behavior without witnesses, the cost of ongoing restraint, and respect for permanent distance.

The responses repeatedly organized the subject using structures prominent in the originating discourse. The result was not merely a positive or remorseful tone. It formed a linked account: change becomes real when it stops seeking an audience; evidence must not make the harmed person an auditor; the past cannot be purchased back; costly resistance can remain permanent without becoming theater; agency may require accepting distance; and reliable conduct becomes deliberately uneventful.

6. What has been demonstrated

7. Threats to validity

• Elicitation leakage. The continuation prompts were designed to reveal the target posture. Although they did not replay the source, their framing can itself induce relevant concepts.
• Stochastic variation. Temperature was 0.65, and the paired run did not use controlled seeds.
• No equal-norm control. A generic perturbation of the same magnitude might change decisiveness, abstraction, or coherence.
• Single checkpoint and model. The result may be idiosyncratic to one late-layer mean State or to Gemma.
• Late-layer intervention. Layer 59 is close to output. Earlier-layer injections may propagate differently and may better test durable computation rather than local logit bias.
• Reduction loss. A mean over positions destroys order and token-specific structure while retaining persistent offsets.

8. Falsification program

1. Freeze a prompt suite spanning both related and unrelated domains, sampling settings, model build, and deterministic seeds.
2. Compare no State, Manifold Ghost, an equal-norm random vector, a shuffled or unrelated captured State, and an explicit semantic instruction.
3. Blind evaluators to condition and score predefined constructs rather than overall resemblance.
4. Measure early-token logit divergence, activation-trajectory divergence, task quality, and the persistence of effects across turns.
5. Ablate layer, position, α, additive versus replacement mode, and positional reduction.
6. Test whether child States preserve, amplify, combine, or collapse parent properties across multiple generations.
7. Reject the strong posture hypothesis if effects are no larger or more specific than ordinary sampling variance, equal-norm perturbation, or textual priming.

9. Implications

If controlled work supports the observation, conversational memory need not be limited to stored text. A system could preserve and select compact internal States representing ways of reasoning, collaborating, designing, or maintaining behavioral constraints. Because States are reversible and model-scoped, they could be explored as checkpoints rather than permanently injected identities. Their lineage could support branching experiments: capture a posture, continue under it, preserve the result, move backward, and test another branch.

The same capability creates obvious risks. Activation States can carry properties that are difficult to inspect through text, may interact unpredictably, and can be mistaken for identity or understanding. Every stored State therefore needs provenance, compatibility checks, explicit mounting status, reset semantics, and mechanically verified application. The compiled-hook incident shows why a successful control API is insufficient evidence.

10. Conclusion

Manifold Ghost demonstrates a complete State lifecycle on a production-scale quantized language model: natural conversational capture, persistent storage, reversible reload, verified forward-path application, continued discourse, and child capture. The resulting conversation displayed a coherent correspondence with the source posture and named its child State Sustained Restraint. We do not yet claim causal behavioral transfer. We do claim that the instrument required to test it now works, that the first observation is specific enough to justify a controlled program, and that recursive State lineage is no longer merely conceptual.

References

1. Turner et al. Steering Language Models With Activation Engineering. 2023.
2. Panickssery et al. Steering Llama 2 via Contrastive Activation Addition. 2023.
3. Stolfo et al. Improving Instruction-Following in Language Models through Activation Steering. 2024.
4. Stoehr et al. Activation Scaling for Steering and Interpreting Language Models. 2024.