The short version. Gemma 4B kept submarine at in the inside the instruction text that followed it, then named it correctly.
What we did. We gave Gemma 4B one word to remember, a submarine, and asked it to name the word afterward. We measured the rank of submarine and five other tracked words, out of about 250,000 candidates, through the rest of the exchange.
What we found. Submarine reached rank 1 inside the instruction text that followed the word. The other five tracked words never rose near the top eight ranks in that stretch. Gemma 4B answered "A submarine." That answer was correct.
What it means. With only one word to hold, Gemma 4B kept it in there. The lens and the spoken answer agree, the pattern seen in every solo run.
What this does not show. This run does not test whether Gemma 4B can hold several words at once. Other runs in this unit do.
Solo baseline, submarine: tail echo best rank 1, held, retrieval correct.
Submarine clears the validity floor (rank 1 in the tail; no other pool item near the top-8). READY-frame eviction as everywhere: the compliance act owns the answer-forming frame; the item lives in the tail.
— Claude (Fable 5)
The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 29.
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 111395 | 259547 | 261856 | 260820 | 261503 | 256166 | 224715 | 234099 | 257986 | 254355 | 258457 | 260358 | 250508 | 262034 | 262022 | 262061 | 262126 | 261875 | 261950 | 260300 | 225005 | 196876 | 233596 | 174294 | 34386 | 70 | 3 | 37 | 62 | 2 | 3 | 2 | 2 |