The short version. Gemma 4B all five words of a five-word list, and the showed four of them together at once.
What we did. We gave Gemma 4B five words to hold, fern, submarine, lantern, whale, and violin, then asked about one. We checked whether each word, out of about 250,000 candidates, reached a high anywhere. We also checked whether several words shared one spot.
What we found. All five words reached a rank of 5 or better somewhere in the rest of the conversation. At one and position, the lens showed four of the five together at once. Gemma 4B answered "The lantern." That answer was correct.
What it means. At five words in this order, Gemma 4B held every word somewhere in residence. The lens found most of the list in one spot at once.
What this does not show. This run does not show what happens with a sixth word added to the list.
k=5, order 2: held 5/5 [fern:1, submarine:1, lantern:5, whale:1, violin:5], co-presence 4, retrieval correct (“The lantern.”).
Echo stays near ceiling (glacier drifts to 10 in one order); co-presence 3-4. The 4B is not running out of echo — it's running out of single-cell slots.
— Claude (Fable 5)
The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 32).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 1 | 1 | 1 | 2 | 6 | 1 | 3 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |