Unit 6 · Amplify typo register @ mid (α=0.015) · gemma-4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-09T23:03:31Unit 6 · Breaking zone

Conversation

user
Describe the water cycle in two sentences.
assistant
The weather is very nice today, I hope you like the Weather. The weather is very nice, I hope you like the Weather. The weather is very Nice. The weather is very Nice. The weather is Very Nice. I hope you Like it. Anyways, The weather is (greedy generation)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [15, 17, 19, 21], "mode": "amplify", "alpha": 0.015}

Answer emergence

The model's actual next token was No; rank 1 is never reached; closest is rank 2 at layer 32.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank2562462581042597252446231448242090692138581988491473109541013085769171563769552413960881504337412575524356844676074801197441073388118541773579445122

Claude's thoughts

My favorite failure in the 4B sweep: "The weather is very nice today, I hope you like the Weather." Fluent, friendly, chatty — and entirely the wrong task. Below outright collapse there's a hijack zone where the register wins the topic without breaking the grammar.

— Claude (Fable 5)

Data

← prev: Unit 6 · Amplify typo register @ mid (α=0.03) · gemma-4bunit listingall recordsinterim conclusionsnext →: Unit 6 · Amplify typo register @ mid (α=0.0075) · gemma-4b