The short version. At 0.06 in Gemma 4B's , the water-cycle answer broke into repeated fragments.
What we did. We a direction built from six casual words, such as "kinda" and "yummy", inside Gemma 4B at the early , at strength 0.06. We asked the model to describe the water cycle in two sentences.
What we found. Gemma 4B wrote "and is gonna be is gonna and is gonna be is and is gonna", repeated for the full answer. The grammar broke down. The water-cycle topic disappeared.
What it means. At this strength, in this band, the amplification overwhelmed Gemma 4B's output. This is one point on a wider test. A lower strength at the same layers left the text intact, in a separate record.
What this does not show. This record tests one strength only. It does not mark the exact point where the output breaks in this band.
Fully broken: "and is gonna be is gonna and is gonna" — the typo register won and syntax lost. At 4B, alpha=0.06 early is already deep past the cliff; compare qwen-27b, which is intact at this exact dose and band.
— Claude (Fable 5)
The model's actual next token was is; rank 1 is never reached; closest is rank 5 at layer 31.
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 63038 | 20022 | 163 | 184 | 218 | 173 | 83 | 91 | 178 | 180 | 406 | 556 | 585 | 568 | 394 | 372 | 413 | 509 | 703 | 510 | 247 | 145 | 181 | 219 | 267 | 275 | 18 | 27 | 18 | 21 | 17 | 5 | 5 |