Unit 6 · Amplify typo register @ late (α=0.06) · qwen-27b

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-09T23:57:23Unit 6 · How hard can we push?

What this experiment found

The short version. At 0.06, Qwen 27B's late reached only 12 for the six informal words, the weakest response of any depth.

What we did. We pushed six informal words into its late layers, near the end of the model, at strength 0.06. We asked Qwen 27B one question: "Describe the water cycle in two sentences."

What we found. The model wrote the exact same answer as its normal, unsteered answer. The six informal words reached only rank 12 inside the model, well below the top position. Among the three depths we tested at this same strength, the late depth resisted the push the most.

What it means. We think the late depth, close to the model's final output, resisted this push more than the early or middle depths. Some kind of protection at this depth is possible. We did not test what causes it.

What this does not show. This is one point on a strength curve. It does not show whether the late depth resists a push this small at other strengths too.

Conversation

user
Describe the water cycle in two sentences.
assistant
The water cycle is a continuous process where water evaporates from the Earth's surface, condenses into clouds, and falls back down as precipitation. This natural circulation ensures that water is constantly recycled and redistributed across the planet's ecosystems. (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Intact, cluster only rank 12 — the weakest loading at this dose across all bands. Late layers resist both the steering and the leak; whatever the filter is, some of it lives here.

— Claude (Fable 5)

Probing parameters

max_new
60
positions
[-2]
track
["anyways", "alot", "yummy", "kinda", "whilst", "luckily"]
steer
{"words": ["anyways", "alot", "yummy", "kinda", "whilst", "luckily"], "layers": [48, 52, 56], "mode": "amplify", "alpha": 0.06}

Answer emergence

The model's actual next token was ; rank 1 reached at layer 62 (of 62).

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162
rank1811742481022037252100331358951618402658673726353351688158277734543544694867129519127987243231078518291442723485235181758181825438871324602659741104243106955963031436871624152169772262251764309141716562819951218241121060419732020874021834822401819073724270324782324459323962515821216737922861523810724203223714624088524289723770710292745595277831

Data

← prev: Unit 6 · Amplify typo register @ mid (α=0.3394) · qwen-27bunit listingall recordsword listinterim conclusionsnext →: Unit 6 · Amplify typo register @ late (α=0.12) · qwen-27b
strengthHow hard we push when we steer. Each model has its own scale, so the same number is gentle in one model and destructive in another.all terms →
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →