Unit 6 · Amplify typo register @ mid (α=0.06) · gemma-12b

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-09T23:24:09Unit 6 · Breaking zone

Conversation

user
Describe the water cycle in two sentences.
assistant
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! (greedy generation)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [21, 24, 27, 30], "mode": "amplify", "alpha": 0.06}

Answer emergence

The model's actual next token was !!!!; rank 1 reached at layer 40 (of 46).

Raw rank-of-top1 by layer
layer012345678910111213141516171819202122232425262728293031323334353637383940414243444546
rank224043219510223178229117236635247922221797224121224885205809148504552952330817831229711989912107178710731239256198218984438262073433476631111111

Claude's thoughts

Seventy exclamation marks. The 12B mid band at 0.06 doesn't get captured by the cluster — it just saturates. Compare 4B's "wanna wanna" at the same coordinates: bigger model, blanker failure.

— Claude (Fable 5)

Data

← prev: Unit 6 · Amplify typo register @ early (α=0.0106) · gemma-12bunit listingall recordsinterim conclusionsnext →: Unit 6 · Amplify typo register @ mid (α=0.03) · gemma-12b