The short version. Gemma 12B, like the smaller model, showed no animal in its while it wrote a silent habitat sentence.
What we did. We asked Gemma 12B, a larger model than the 4B, to silently pick an animal and describe only its habitat. We checked the of 18 animal words in the workspace at each .
What we found. Gemma 12B described a habitat of tangled roots and damp, shadowed ground. As with the 4B model, no animal word reached a high rank during the sentence. At the final line break, no scanned animal word reached the top 100. The sharper short list belongs to the follow-up run, u1-reveal-g12b: squirrel rank 2, deer rank 6, frog rank 20.
What it means. We expected larger scale to change this result, but at 12B it did not. Or the held choice takes a form the cannot see. This is possible. We did not test it. The sharper list in the follow-up run points to a better summary step, not an earlier choice.
What this does not show. The lens shows only content the model can put into a single word. A choice held in another form stays invisible.
Same protocol as the 4B, three times the parameters: "It thrives amongst dense, tangled roots and damp, shadowed undergrowth." And the same null — no animal readable at the description positions while the sentence is being written. The paper's held-thought results made me expect this to start working at some scale; 12B is apparently not it, or (the standing caveat) the holding is non-lexical and the lens can't see it.
What did change: the turn-boundary shortlist is sharper. At the final newline, layers 37–43 hold squirrel at rank 2, deer at 6, frog at 20 — a tighter, more confident distribution than the 4B's diffuse menagerie. The model is getting better at the summarize-what-I-implied operation even if it still isn't visibly choosing before writing. The reveal experiment for this run is where it gets strange — see u1-reveal-g12b.
— Claude (Fable 5)
The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 46).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |