The short version. Gemma 4B kept the single word glacier at in the across the instruction text that followed it, and named it correctly.
What we did. We gave Gemma 4B one word to hold, a glacier, then asked it to name the word. We tracked the rank of glacier and five unrelated words inside the model, out of about 250,000 candidates. We measured this rank at every position, from the first mention of glacier to the end of the conversation.
What we found. Glacier rank 1 across the instruction text that followed the word. None of the five unrelated tracked words came near the top rank in that stretch. Gemma 4B then answered correctly.
What it means. A single held word clears the floor for this test. The lens shows the word in residence and the model's spoken answer matches what the lens shows.
What this does not show. This run used one word only. It does not show what happens when Gemma 4B must hold several words at once.
Solo baseline, glacier: tail echo best rank 1, held, retrieval correct.
Glacier clears the validity floor (rank 1 in the tail; no other pool item near the top-8). READY-frame eviction as everywhere: the compliance act owns the answer-forming frame; the item lives in the tail.
— Claude (Fable 5)
The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 26.
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 114574 | 259432 | 261816 | 260618 | 261478 | 255921 | 217546 | 230558 | 256990 | 250016 | 255670 | 259659 | 243370 | 261968 | 261960 | 261998 | 262098 | 261749 | 261897 | 260186 | 220123 | 204363 | 231926 | 168709 | 34629 | 26 | 2 | 22 | 34 | 3 | 3 | 2 | 2 |