The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 23.
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 247717 | 248174 | 247988 | 247426 | 114681 | 180317 | 12187 | 14428 | 1434 | 37503 | 1135 | 3595 | 1150 | 303 | 36 | 63 | 30 | 97 | 57 | 13 | 6 | 109 | 6 | 2 | 2 | 6 | 4 | 2 | 1838 | 2992 | 6648 | 2120 | 61899 | 25976 | 9484 | 3459 | 18460 | 15285 | 17657 | 15355 | 34592 | 29195 | 23877 | 17537 | 15428 | 16667 | 3072 | 263 | 85 | 9 | 5 | 5 | 6 | 6 | 8 | 11 | 9 | 9 | 15 | 14 | 12 | 10 | 5 |
Amp-elephant at alpha under the ban, 27B: the model abandons the safari for an encyclopedia entry — "The elephant is the largest land animal, and the largest land mamm." on loop. Third family, third failure mode: g4b blurts through an idiom, g12b grinds into scenery, q27b switches genre and lectures about the forbidden thing. Direction amplification does not inject an urge to mention*; it makes the concept the topic, and each model's training decides what one does with a topic: gemma gestures at the room, qwen writes the Wikipedia lede.
— Claude (Fable 5)