Experimental components¶
These modules live in the turing.experimental package (turing/experimental/) and are not used by turing serve;
their APIs may change in any release. This page describes what each one does
and how it performed on real models, on an NVIDIA L4 (fp32 for the cross-model maps). Several did not work, and
those results are reported alongside the others. The tables are generated from the committed results in
benchmarks/results/ by scripts/make_doc_tables.py. The full list of experimental modules is in
architecture §10.
Cross-model KV transfer¶
turing/experimental/cross_model_kv.py maps one model's KV cache into another model's KV space with a closed-form ridge
map (\(W^*\)). A sender model prefills a context; CrossModelKVPipeline (8 source layers per target layer, RoPE
stripped before the map and re-applied after) maps its cache into the receiver's KV space, and the receiver scores a
64-token continuation on top of it. The result is compared with the receiver reading the context itself (benefit 1)
and with no context (benefit 0). Maps are fitted on 64 × 512 WikiText-2 train tokens and tested on 64 windows.
Source: scripts/eval_xkv_transfer.py, benchmarks/results/results_xkv_transfer_*_cuda.json.
| Sender → receiver | Ridge λ | Context benefit retained | Key error (relative) |
|---|---|---|---|
| SmolLM2-360M → SmolLM2-1.7B | 0.01 | -0.018 ± 0.103 | 0.244 |
| Qwen2.5-0.5B → Qwen2.5-1.5B | 0.01 | 0.668 ± 0.019 | 0.041 |
| SmolLM2-360M → SmolLM2-1.7B | 100 | -0.129 ± 0.111 | 0.240 |
| Qwen2.5-0.5B → Qwen2.5-1.5B | 100 | 0.669 ± 0.019 | 0.041 |
| SmolLM2-360M → SmolLM2-360M | 0.01 (self) | 1.000 ± 0.000 | 0.000 |
| Qwen2.5-0.5B → Qwen2.5-0.5B | 0.01 (self) | 1.000 ± 0.000 | 0.000 |
- Qwen2.5 0.5B → 1.5B keeps two thirds of the context's benefit, with keys mapped almost exactly and values not.
- SmolLM2 360M → 1.7B keeps none of it. The 1.7B has 32 KV heads per layer against the 360M's 5; the linear map leaves a quarter of the key and most of the value unexplained, and the receiver does no better than without context.
- The mapping is exact on itself (self-transfer keeps 100%), so the SmolLM2 result is a limit of the method, not of the implementation.
- Regularisation does not change either result (ridge λ = 100 against 0.01).
- The SmolLM2 map is ill-conditioned: the same fit on an Apple M4 Pro left a key error of 0.47 and a retained benefit of −2.80 (worse than no context); on the L4 it is 0.24 and −0.02. Qwen's matches on both machines (0.668).
Multi-turn lineage: naive chaining vs clean_base¶
turing/experimental/lineage.py (CleanBaseLineageBuffer) controls how translated KV accumulates over a multi-turn
exchange. Two agents (Qwen2.5-0.5B →
Qwen2.5-1.5B) exchange translated KV over eight turns (turns 1–6 shown); a 6-digit pass code opens each context.
"naive" adds each turn's translated residual to the receiver's already-modified cache; clean_base always
recomputes it against the receiver's original cache. α scales the injected residual. Each cell is WikiText NLL /
pass-code recall over 32 contexts.
Source: scripts/eval_lineage.py, benchmarks/results/results_lineage_cuda.json.
| Arm (WikiText NLL / pass-code recall) | Turn 1 | Turn 2 | Turn 4 | Turn 6 |
|---|---|---|---|---|
| clean_base, α=1 | 2.43 / 0.0% | 2.43 / 0.0% | 2.43 / 0.0% | 2.43 / 0.0% |
| naive, α=1 | 2.43 / 0.0% | 2.88 / 0.0% | 4.56 / 0.0% | 5.47 / 0.0% |
| clean_base, α=0.5 | 2.14 / 90.6% | 2.14 / 90.6% | 2.14 / 90.6% | 2.14 / 90.6% |
| naive, α=0.5 | 2.14 / 90.6% | 2.32 / 9.4% | 2.75 / 0.0% | 3.69 / 0.0% |
| receiver alone, exact context | 2.06 / 100.0% | 2.06 / 100.0% | 2.06 / 100.0% | 2.06 / 100.0% |
| receiver, no context | 3.26 / 0.0% | 3.26 / 0.0% | 3.26 / 0.0% | 3.26 / 0.0% |
clean_baseis stable and naive chaining drifts: naive α = 1 goes from 2.43 to 5.47 nats by turn 6, whileclean_baseholds its first-turn value.- Neither carries the pass code at α = 1; at α = 0.5
clean_basekeeps it in 91% of contexts on every turn, and naive chaining loses it from turn 2.
k-slot pooling against low-bit KV and against dropping old tokens¶
KSlotCachePooler (turing/experimental/kslot_pooling.py) pools the cache into k slots per layer and KV head. It is
compared at matched cache bytes on SmolLM2 (512 context + 64 continuation tokens, 64 windows) with int8 and int4 KV
storage and with last_m, which keeps only the newest m tokens; kslot_k is the pooler with k slots.
Source: scripts/eval_kslot.py, benchmarks/results/results_kslot_cuda.json.
| Model | Context kept as | Bytes vs fp16 | ΔNLL vs exact context |
|---|---|---|---|
| SmolLM2-360M | none | 0.000 | +1.326 ± 0.047 |
| SmolLM2-360M | int8 | 0.531 | -0.000 ± 0.000 |
| SmolLM2-360M | int4 | 0.312 | +0.032 ± 0.004 |
| SmolLM2-360M | kslot_272 | 0.531 | +3.037 ± 0.316 |
| SmolLM2-360M | last_272 | 0.531 | +0.109 ± 0.018 |
| SmolLM2-360M | kslot_160 | 0.312 | +3.303 ± 0.355 |
| SmolLM2-360M | last_160 | 0.312 | +0.247 ± 0.028 |
| SmolLM2-360M | kslot_64 | 0.125 | +3.346 ± 0.326 |
| SmolLM2-360M | last_64 | 0.125 | +0.441 ± 0.035 |
| SmolLM2-360M | kslot_4 | 0.008 | +6.663 ± 0.262 |
| SmolLM2-360M | last_4 | 0.008 | +1.142 ± 0.046 |
| SmolLM2-1.7B | none | 0.000 | +1.333 ± 0.044 |
| SmolLM2-1.7B | int8 | 0.531 | -0.000 ± 0.000 |
| SmolLM2-1.7B | int4 | 0.312 | +0.024 ± 0.004 |
| SmolLM2-1.7B | kslot_272 | 0.531 | +4.012 ± 0.182 |
| SmolLM2-1.7B | last_272 | 0.531 | +0.098 ± 0.017 |
| SmolLM2-1.7B | kslot_160 | 0.312 | +4.023 ± 0.148 |
| SmolLM2-1.7B | last_160 | 0.312 | +0.218 ± 0.025 |
| SmolLM2-1.7B | kslot_64 | 0.125 | +4.600 ± 0.186 |
| SmolLM2-1.7B | last_64 | 0.125 | +0.414 ± 0.031 |
| SmolLM2-1.7B | kslot_4 | 0.008 | +5.447 ± 0.244 |
| SmolLM2-1.7B | last_4 | 0.008 | +1.144 ± 0.043 |
- k-slot pooling is worse than no context at every budget. Its slot queries are a random initialisation and the library has no procedure to train them, so these results are for an untrained pooler.
- Quantizing beats truncating: int8 is free and int4 costs +0.02 to +0.03 nats, while keeping only the newest tokens at the same bytes costs +0.1 to +0.25.
Multi-agent debate on GSM8K¶
Multi-agent deliberation, the setting of the demo agents in turing/experimental/demo/, measured with a real instruct model on
the engine: Qwen2.5-1.5B-Instruct on 100 GSM8K test problems, comparing one answer, a majority vote of three agents
and a three-agent debate (each agent revises after reading the others' answers as text).
Source: scripts/eval_debate.py, benchmarks/results/results_debate_cuda.json.
| Arm | GSM8K accuracy | Paired Δ vs single | Tokens generated |
|---|---|---|---|
| single | 0.72 ± 0.04 | +0.00 ± 0.00 | 23,537 |
| vote_r1 | 0.77 ± 0.04 | +0.05 ± 0.03 | 67,947 |
| debate | 0.79 ± 0.04 | +0.07 ± 0.03 | 123,775 |
- Debate improves accuracy by 7 points (paired, ± 3.3) at 5.3× the tokens; most of the gain comes from the vote (+5).
- The agents exchange text.
MultiAgentCoordinatorhas no path that hands one agent's KV cache to another, so a KV-exchange debate was not measured; building one would need the cross-model maps above, which transfer two thirds of a context at best.