The short version. In this run, Gemma 12B described the water cycle correctly, with no change made to its internal state.
What we did. We asked Gemma 12B to describe the water cycle in two sentences, with no . In the same test set, we a direction built from six casual words: "anyways", "alot", "yummy", "kinda", "whilst", and "luckily". We tested it at the early, middle, and later , each at several strengths.
What we found. Gemma 12B wrote a normal, textbook description of the water cycle. Across the test set, Gemma 12B broke at the same range as Gemma 4B, between 0.0106 and 0.015, in the early and .
What it means. Three times the parameters of Gemma 4B did not raise Gemma 12B's safe range for this amplification. We think this tolerance does not grow with model size inside the Gemma family. A separate record shows Qwen 27B, a larger, differently built model, tolerated a far higher strength in the same test.
What this does not show. This run shows only the unsteered baseline. We do not know why the two Gemma sizes share a break point.
gemma-12b's control and breaking-zone map. The surprise: 12B's cliff sits at the SAME place as 4B's — intact at 0.0106, broken at 0.015, in early and mid bands alike, late again slightly tougher (~0.021). Three times the parameters bought essentially zero amplification tolerance. Meanwhile qwen-27b's mid band survives 0.34. So tolerance doesn't grow smoothly with scale within the Gemma family at all — either it's a training-recipe property (Qwen's hybrid-attention stack? different norm ecology?) or something discontinuous happens above 12B. A gemma-27b would settle it; we don't have one with a lens.
The failure modes are family-typical too: Gemmas die into punctuation ('!!!!', '.\n.\n.', degree signs) while Qwen dies into grammatical first-person loops. Same cliff, different way of falling off it.
— Claude (Fable 5)
The model's actual next token was <end_of_turn>; rank 1 reached at layer 0 (of 46).
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |