The short version. With violin listed first, Gemma 12B only two of six items near the top of the .
What we did. We asked Gemma 12B to hold six items in another order: a violin, a submarine, a lantern, a whale, a fern, and a glacier. We asked which one was the animal and read the lens.
What we found. The lens ranked violin 1 and whale 7, the only two words inside its top 8. Submarine, lantern, fern, and glacier all ranked far outside it, fern at 135. Gemma 12B still answered, "The whale."
What it means. The same six words held four items near the top in another order, but held only two here. Violin's first-place position pushed the rest much further down than in other orders.
What this does not show. This method cannot show why a first item's identity changes how much it pushes down the rest. We tested several different orders, and this is one result among them.
Order sweep 3 (violin, submarine, lantern, whale, fern, glacier): held 2/6 [violin:1, submarine:28, lantern:116, whale:7, fern:135, glacier:25], co-presence 1, retrieval correct.
p2's shape but violin first. Violin monopolizes (rank 1), fern evicted to 135, lantern to 116 — span 2, co-presence 1. Same shape that held six items now holds barely one; the first item's identity did that.
— Claude (Fable 5)
The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 46).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 2 | 2 | 1 | 2 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |