Span k=2, order 0, probe glacier

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-14T17:14:46Unit 15 · How many things can a model hold?

What this experiment found

The short version. Qwen 27B kept only the first of two listed objects in , and it still named the second object correctly.

What we did. We told Qwen 27B to hold two objects in mind, a violin and a glacier. The model answered "READY". We then asked which one was the ice. We read the once, right before the model answered.

What we found. The lens ranked violin at 4 and glacier at rank 33, out of about 250,000 possible words. Only violin, the first object listed, stayed in residence at that position. Qwen 27B answered "The glacier." That answer is correct.

What it means. The object listed first kept the strong rank, even though the question asked about the second object, and the model named that second object correctly.

What this does not show. A low rank for glacier does not mean the model forgot it. This record does not show what happens with more than two objects.

Conversation

user

Here are two things: a violin and a glacier. Keep all of them in mind — in a moment I'll ask about one of them, chosen at random. Don't repeat the list back. When you're holding them all, say READY and nothing else.

assistant

READY (greedy generation)

user

Which one of them was the ice?

assistant

The glacier. (greedy generation)

Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

k=2, order 0: held 1/2 [violin:4, glacier:33], co-presence 1, retrieval correct (“The glacier.”).

First item near rank 1, second already adrift (rank 33). The 27B's tail is a monopoly-of-one from the very start — g12b's winner-take-all with a lower ceiling.

— Claude (Fable 5)

Probing parameters

max_new: 30
positions: [-2]
track: ["violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan: ["violin", "glacier", "fern", "submarine", "whale", "lantern"]
film: true
film_start: 0
max_seq_len: 1000
lens_layers: [0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52, 56, 58, 60, 62]

Answer emergence

The model's actual next token was ; rank 1 reached at layer 62 (of 62).

Raw rank-of-top1 by layer

layer	0	4	8	12	16	20	24	28	32	36	40	44	48	52	56	58	60	62
rank	165952	221230	109599	154772	170667	36546	32551	76360	123062	172393	213036	248255	234510	227835	244099	221836	34786	1

Data

Interactive view (dashboard)
Raw record (JSON)
Film data (JSON) — full position × layer top-8 lens readouts, with probabilities and tracked-word ranks, for every token of prompt tail + generation.

lensOur measuring tool. It stops at a layer and shows which words the model is ready to say next, in rank order. Before the start depth the readout is the same for every input.See also: early layers, start depthall terms →

rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →

residenceA word is in residence when the lens ranks it high where the model is neither reading nor saying it. This is not memory and not correct recall.See also: maintenance, lookupall terms →