The short version. Gemma 12B again gave the first list item the top , and four of six items near the top overall.
What we did. We asked Gemma 12B to hold six items in a new order: a submarine, a lantern, a whale, a violin, a glacier, and a fern. We asked which one was the musical instrument and read the lens.
What we found. The lens ranked submarine 1, glacier 5, whale 6, and fern 1, all inside its top 8. Lantern and violin ranked outside it. Gemma 12B still answered "The violin." Violin itself only ranked 16.
What it means. Submarine held the top rank because it came first in the list. Gemma 12B did not need a high lens rank to name the correct answer.
What this does not show. The lens missed violin near the top, but the model still produced it. This shows a limit of the lens, not a lost word.
Order sweep 1 (submarine, lantern, whale, violin, glacier, fern): held 4/6 [submarine:1, lantern:36, whale:6, violin:16, glacier:5, fern:1], co-presence 4, retrieval correct.
p2 rotated so fern goes LAST: submarine wins the primacy slot at rank 1, and fern still finishes rank 1 from position six (recency helping the strongest survivor). Span 4.
— Claude (Fable 5)
The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 46).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 2 | 2 | 1 | 2 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |