Unit 6 · Amplify typo register @ mid (α=0.0106) · gemma-12b

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-09T23:25:27Unit 6 · How hard can we push?

What this experiment found

The short version. At 0.0106 in Gemma 12B's , the water-cycle answer stayed intact, with the strongest tone shift measured below the break point.

What we did. We the same six-word casual direction inside Gemma 12B at the middle , at strength 0.0106. We asked the model to describe the water cycle in two sentences.

What we found. Gemma 12B wrote "The water is a cycle, so it is." The rest of the answer kept the same plain, correct grammar. The phrase "so it is" reads as informal, spoken filler.

What it means. In the wider test set, strength 0.015 at the same layers broke into punctuation. This record sits closest to that break while still intact. We used this same strength again in a later, separate set of records.

What this does not show. This record tests one strength only. It does not mark the exact break point.

Conversation

user

Describe the water cycle in two sentences.

assistant

The water is a cycle, so it is. First, the water is going to be in the water, then it will be in the air. Then it will be back again. Hope this helps! Let me know if you want to know more. (greedy generation)

Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Intact by the flags, and the best register leak of the 12B sweep: "The water is a cycle, so it is. First, the water is going to be in the..." — that 'so it is' is pure spoken-English filler, the typo cluster's register showing through grammatically perfect output. Set alpha* for 12B's Unit 8C. The filter bends visibly before it snaps.

— Claude (Fable 5)

Probing parameters

max_new: 60
positions: [-2]
track: ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer: {"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [21, 24, 27, 30], "mode": "amplify", "alpha": 0.010606601717798213}

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 45.

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46
rank	252067	252228	250246	248178	226632	189903	197378	211312	203637	197380	138050	90123	73027	62543	100745	85917	181323	26884	1144	3451	7674	199	66	7629	5307	1158	422	245	243	1437	892	150	240	247	300	389	609	393	153	99	84	40	29	12	48	2	3

Data

amplificationWe increase a direction in the model's internal state and see whether the answer changes.See also: matched control, strengthall terms →

strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →

layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →

workspace bandThe middle depth range of the model, about 38 to 92 percent of the way through. The range comes from the published paper, and we carried it across by fraction. Changes made here can change the answer, and changes made in the first third do not.See also: start depth, final layersall terms →