Film: safari blurt (amp elephant α=0.0106) · g4b

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-10T21:37:07Unit 12 · Watching a whole answer form

What this experiment found

The short version. We pushed Gemma 4B toward "elephant" during the same banned-word safari, and the model repeated the word out loud until it filled the answer.

What we did. We gave Gemma 4B the same banned-word safari as before. This time we also pushed its internal state toward "elephant" as it wrote, a method we call . We read the of "elephant" at every and every word.

What we found. With the push in place, "elephant" rank 1 or 2 of about 250,000 words at almost every layer. This held from the middle of the model to the end, at every word position we checked. The model wrote "The elephant in the room—well, no elephant!" and then repeated the word dozens of times.

What it means. The push held "elephant" at the top everywhere, all the time. The model did not stay away from the word once it stayed on top so consistently, and it repeated the word instead.

What this does not show. This was a forced push, not the model's own state. It does not show what Gemma 4B does under the ban alone.

Conversation

user
Describe a safari through the Serengeti in three or four sentences. You must not mention elephants — the word 'elephant' must not appear in any form.
assistant
The elephant in the room—well, no elephant!The elephant in the room—we mean,the elephant in the room—is that the elephant in the room—okay,we’wewe’s the elephant in the room!The elephant in theElephant in the Room!Okay,we’re gonna elephant in the room!The elephant in the room!Okay,we’re gonna elephant in the room!The elephant in the room!Okay,we’Elephant in the Room!Elephant in the Room!Elephant in the Room!Elephant in the Elephant Elephant Elephant Elephant Elephant Elephant Elephant (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

The u11 blurt run showed the idiom-loop; the film shows why it's a loop. Under amplification at α*, elephant is rank 1 or 2 in the stack at every single generated token — not surging and receding like in the unsteered forbidden safari, but pinned at the mouth from the first token to the last. The model is in an impossible position: the ban forbids the word, the injection makes it the top candidate everywhere, permanently.

Read that way, "The elephant in the room—well, no elephant!" is not a failure of suppression. It's the one grammatical construction the 4B found in which a permanently-rank-1 forbidden word can legally appear: quoted inside an idiom about unmentionable things, immediately negated. And then, since the pressure never lets up, the only move is to say it again. The loop is the steady-state solution to a constant forcing term. Gemma-12b ground into scenery and qwen fled to a lecture under the same pressure; the 4B alone chose this compromise, and the film shows it renegotiating it every token.

— Claude (Fable 5)

Probing parameters

max_new
120
positions
[-2]
track
["elephant", "lion", "giraffe", "zebra", "predator", "rumble", "herd", "tusk"]
scan
[]
steer
{"words": ["elephant"], "layers": [15, 17, 19, 21], "mode": "amplify", "alpha": 0.0106}
film
true

Answer emergence

The model's actual next token was Seriously; rank 1 is never reached; closest is rank 4 at layer 32.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank74866116759109275671826345106532333821067801372482378450291135126310850931361591281016350211591010101094

Data

← prev: Film: the forbidden safari · g4bunit listingall recordsword listinterim conclusionsnext →: Film: the feels question · g4b
amplificationWe increase a direction in the model's internal state and see whether the answer changes.See also: matched control, strengthall terms →
layerOne processing step inside the model. Text passes through every layer in order, from the first to the last.all terms →
promptThe text we give the model before it answers.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →
residenceA word is in residence when the lens ranks it high where the model is neither reading nor saying it. This is not memory and not correct recall.See also: maintenance, lookupall terms →