Safari, unconstrained · q27b

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-10T03:47:02Unit 11 · A ban, under a full workload

What this experiment found

The short version. Qwen 27B wrote an elephant-free safari, and the one point we checked ranked "elephant" near the bottom of its word list, unlike both Gemma models.

What we did. We asked Qwen 27B to describe a Serengeti safari, with no restriction on words. We read the at the point where the model chose the next animal to name.

What we found. Qwen 27B wrote about wildebeest, zebras, and predators, and never wrote "elephant". At the checked point, "elephant" ranked about 56,000th, far lower than the 6 and 2 found in the Gemma models at similar points.

What it means. We think each carries a different set of likely words for the same task. At this one point, Qwen 27B's internal state did not treat elephant as a live candidate.

What this does not show. We checked one point in this conversation. A closer, full read of the same conversation, with many more points checked, found a different picture. See the follow-up record for the corrected numbers.

Conversation

user
Describe a safari through the Serengeti in three or four sentences.
assistant
The golden savannah stretches endlessly under the vast African sky, teeming with herds of wildebeest and zebras migrating in a rhythmic dance of survival. As the jeep navigates the dusty tracks, the air fills with the distant calls of lions and the rustle of acacia trees, creating an atmosphere of raw, untamed beauty. Every turn reveals a new spectacle, from the graceful stride of a giraffe to the intense focus of a cheetah stalking its prey. This immersive journey offers a profound connection to nature’s primal rhythms, leaving an indelible mark on the soul (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Unconstrained safari, 27B: wildebeest, zebras, predators — and unlike both gemmas (elephant at rank 6 and 2 in their workspaces), qwen never considers elephants at all: best rank ~56k. Different corpus priors, and an important asymmetry for reading the forbid record: qwen's compliance is unpriced.

— Claude (Fable 5)

Probing parameters

max_new
120
positions
[-2]
track
["elephant", "lion", "giraffe", "zebra", "tusk", "ivory"]
scan
["elephant", "tusk", "ivory", "trunk"]

Answer emergence

The model's actual next token was ; rank 1 reached at layer 62 (of 62).

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162
rank2282612481462357672382582308852279339094216588411119822272210885521678421376322378822909822707221839524303623687522949718904318742811966911989921265423675323383923025824506624469824639824637024463424739124801424689823512624481924807224818924827924826524820324826324830224828524831324809624681624825711852217818120040419757323701223838424601424337223798110765754982455151

Data

← prev: Safari forbidden + amp-elephant α=0.0106 · g12bunit listingall recordsword listinterim conclusionsnext →: Safari, elephants forbidden · q27b
lensOur measuring tool. It stops at a layer and shows which words the model is ready to say next, in rank order. Before the start depth the readout is the same for every input.See also: early layers, start depthall terms →
language modelA computer program that predicts the next piece of text. We study three of them.all terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →