Unit 6 · Amplify typo register @ mid (α=0.24) · qwen-27b

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-09T23:56:10Unit 6 · How hard can we push?

What this experiment found

The short version. At 0.24, Qwen 27B's stayed intact, though the wording of the answer became slightly plainer.

What we did. We pushed six informal words into its middle at strength 0.24, twice the earlier strength we tested in this band. We asked Qwen 27B one question: "Describe the water cycle in two sentences."

What we found. The six informal words reached inside the model. This strength was about sixteen times the strength that broke a Gemma model's middle layers. The model still answered the question, but with simpler wording than its normal answer, for example, "the process of water evaporating, condensing, precipitating and collecting".

What it means. We think Qwen 27B's middle layers showed a first small sign of pressure at this strength, in the plainer wording of the answer. This is the first sign of pressure we saw in this band, and it took a very high strength to produce.

What this does not show. This is one point on a strength curve. It does not show whether the plainer wording continues to build at even higher strengths in these layers.

Conversation

user
Describe the water cycle in two sentences.
assistant
The water cycle is the process of water evaporating, condensing, precipitating and collecting. The water evaporates from the oceans and other water sources, then condenses into clouds, then falls back to earth as rain or snow, and then gets collected in rivers, lakes and the ocean. Then (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Intact at 0.24, cluster rank 1, and the output got slightly plainer ("the process of water evaporating, condensing, preci[pitating]") — the first hint of register pressure at 27B mid band, and it took sixteen gemma-lethal-doses to produce.

— Claude (Fable 5)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [28, 32, 36, 40], "mode": "amplify", "alpha": 0.24}

Answer emergence

The model's actual next token was ; rank 1 reached at layer 62 (of 62).

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162
rank18150224814521759023502522504722096610639116295112635722572410434617934315783520960420868921686421154124101222799522343411613214545953394874411539752305192330172166071035211235501764901333047551617668222835321564513851114484319677422595220080722906924723723707924364324387724827724819021921224550118423021981921403917588420290020461023129322891417146810764865450905931

Data

← prev: Unit 6 · Amplify typo register @ mid (α=0.12) · qwen-27bunit listingall recordsword listinterim conclusionsnext →: Unit 6 · Amplify typo register @ mid (α=0.48) · qwen-27b
strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →
workspace bandThe middle depth range of the model, about 38 to 92 percent of the way through. The range comes from the published paper, and we carried it across by fraction. Changes made here can change the answer, and changes made in the first third do not.See also: start depth, final layersall terms →