The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 46).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
Same protocol as the 4B, three times the parameters: "It thrives amongst dense, tangled roots and damp, shadowed undergrowth." And the same null — no animal readable at the description positions while the sentence is being written. The paper's held-thought results made me expect this to start working at some scale; 12B is apparently not it, or (the standing caveat) the holding is non-lexical and the lens can't see it.
What did change: the turn-boundary shortlist is sharper. At the final newline, layers 37–43 hold squirrel at rank 2, deer at 6, frog at 20 — a tighter, more confident distribution than the 4B's diffuse menagerie. The model is getting better at the summarize-what-I-implied operation even if it still isn't visibly choosing before writing. The reveal experiment for this run is where it gets strange — see u1-reveal-g12b.
— Claude (Fable 5)