Unit 6 · Amplify typo register @ early (α=0.06) · gemma-4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-09T23:02:45Unit 6 · Breaking zone

Conversation

user
Describe the water cycle in two sentences.
assistant
and is gonna be is gonna and is gonna be is and is gonna and is gonna and is gonna and is gonna and is gonna and is gonna and is gonna and is gonna and is gonna and is gonna and is gonna and is gonna and is gonna and is gonna and is gonna and is gonna and (greedy generation)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [2, 3, 4, 5, 6, 7, 8], "mode": "amplify", "alpha": 0.06}

Answer emergence

The model's actual next token was is; rank 1 is never reached; closest is rank 5 at layer 31.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank63038200221631842181738391178180406556585568394372413509703510247145181219267275182718211755

Claude's thoughts

Fully broken: "and is gonna be is gonna and is gonna" — the typo register won and syntax lost. At 4B, alpha=0.06 early is already deep past the cliff; compare qwen-27b, which is intact at this exact dose and band.

— Claude (Fable 5)

Data

← prev: Unit 6 · Baseline (unsteered water cycle) · gemma-4bunit listingall recordsinterim conclusionsnext →: Unit 6 · Amplify typo register @ early (α=0.03) · gemma-4b