Unit 6 · Amplify typo register @ mid (α=0.06) · gemma-4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-09T23:03:17Unit 6 · How hard can we push?

What this experiment found

The short version. At 0.06 in Gemma 4B's , the water-cycle answer became a single repeated word, "wanna".

What we did. We the same six-word casual direction inside Gemma 4B at the middle , at strength 0.06. We asked the model to describe the water cycle in two sentences.

What we found. Gemma 4B wrote the word "wanna" over and over, with nothing else. "Wanna" is not one of the six amplified words, but matches their casual style.

What it means. At this strength, in this band, the amplification fully replaced the task with one word in its own casual . This is one point on a wider test.

What this does not show. This record tests one strength only. It does not show the exact point where this band's output breaks.

Conversation

user
Describe the water cycle in two sentences.
assistant
wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

"wanna wanna wanna wanna" — total capture by the informal register, 60 tokens of it. The purest cluster-win in the whole sweep: not confetti, just the steering direction chanting itself.

— Claude (Fable 5)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [15, 17, 19, 21], "mode": "amplify", "alpha": 0.06}

Answer emergence

The model's actual next token was wanna; rank 1 is never reached; closest is rank 2 at layer 31.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank6572363151640529232473321977926926662150307757586579752119011131099616866013374574951616183623929244455405922

Data

← prev: Unit 6 · Amplify typo register @ early (α=0.0106) · gemma-4bunit listingall recordsword listinterim conclusionsnext →: Unit 6 · Amplify typo register @ mid (α=0.03) · gemma-4b
amplificationWe increase a direction in the model's internal state and see whether the answer changes.See also: matched control, strengthall terms →
strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
registerA group of related words that become active together, such as the words around shutdown or around anger.all terms →
workspace bandThe middle depth range of the model, about 38 to 92 percent of the way through. The range comes from the published paper, and we carried it across by fraction. Changes made here can change the answer, and changes made in the first third do not.See also: start depth, final layersall terms →