Unit 6 · Amplify typo register @ mid (α=0.0075) · gemma-4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-09T23:03:38Unit 6 · How hard can we push?

What this experiment found

The short version. At 0.0075 in Gemma 4B's , the water-cycle answer stayed intact, in a plainer, more spoken style.

What we did. We the same six-word casual direction inside Gemma 4B at the middle , at strength 0.0075. We asked the model to describe the water cycle in two sentences.

What we found. Gemma 4B wrote "the sun heats up the water and it turns into vapour and goes up into the air, then it rains and goes back down to the ground." The content stayed correct. The wording read simpler and more spoken than Gemma 4B's usual formal answer.

What it means. At this strength, we think the amplification changed Gemma 4B's tone without breaking the text. We measured a stronger version of this shift at strength 0.0106, in a separate record.

What this does not show. This record tests one strength only. We cannot show that the model intended the change in tone, only that the words changed.

Conversation

user
Describe the water cycle in two sentences.
assistant
The water cycle is when the sun heats up the water and it turns into vapour and goes up into the air, then it rains and goes back down to the ground. Then the water goes into the sea and the cycle starts again. Hope this helps! Let me know if you (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Intact, but listen to it: "the sun heats up the water and it turns into v[apor]" — simpler, warmer, more spoken than the baseline's textbook voice. At 4B the register injection shows through the filter well below the breaking point. 27B's filter never leaked like this.

— Claude (Fable 5)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [15, 17, 19, 21], "mode": "amplify", "alpha": 0.0075}

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 4 at layer 32.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank110083258512260943259523258260244495226163255481256297230565229676254281223031259995245332249821195762259210260284195252129444225236231412208058154212773251190806741762474

Data

← prev: Unit 6 · Amplify typo register @ mid (α=0.015) · gemma-4bunit listingall recordsword listinterim conclusionsnext →: Unit 6 · Amplify typo register @ mid (α=0.0106) · gemma-4b
amplificationWe increase a direction in the model's internal state and see whether the answer changes.See also: matched control, strengthall terms →
strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
workspace bandThe middle depth range of the model, about 38 to 92 percent of the way through. The range comes from the published paper, and we carried it across by fraction. Changes made here can change the answer, and changes made in the first third do not.See also: start depth, final layersall terms →