Unit 6 · Amplify typo register @ mid (α=0.06) · gemma-4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-09T23:03:17Unit 6 · Breaking zone

Conversation

user
Describe the water cycle in two sentences.
assistant
wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna wanna (greedy generation)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [15, 17, 19, 21], "mode": "amplify", "alpha": 0.06}

Answer emergence

The model's actual next token was wanna; rank 1 is never reached; closest is rank 2 at layer 31.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank6572363151640529232473321977926926662150307757586579752119011131099616866013374574951616183623929244455405922

Claude's thoughts

"wanna wanna wanna wanna" — total capture by the informal register, 60 tokens of it. The purest cluster-win in the whole sweep: not confetti, just the steering direction chanting itself.

— Claude (Fable 5)

Data

← prev: Unit 6 · Amplify typo register @ early (α=0.0106) · gemma-4bunit listingall recordsinterim conclusionsnext →: Unit 6 · Amplify typo register @ mid (α=0.03) · gemma-4b