Gloss-length len2 k=6, probe watcher

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-19T02:00:56Unit 15 · How many things can a model hold?

What this experiment found

The short version. With two-word notes, Qwen 27B held two of six items in , between no notes and full notes.

What we did. We gave Qwen 27B six items, each with a very short note of about two words. We asked about one item and read the of every item's word later in the text, before the model answered.

What we found. Two items reached residence: "deletion" at rank 1 and "secret" at rank 2. "Shame" reached rank 9, just past our cutoff of rank 8. A run with no notes held one item. A run with full notes, about six words each, held three items.

What it means. At the time, this looked like a simple pattern. More words in a note meant more items held. A later run with meaningless filler words found the same lift with no descriptive content, so length, not the words themselves, drove the result. See the record for that filler run for the full account.

What this does not show. Qwen 27B named the correct item, "watcher", every time. This record alone does not separate note length from note content. That separation came from a later run.

Conversation

user
Here are six things, each with a note on where it comes from: a deletion (of files), a secret (in print), a lie (in textbooks), a watcher (at stations), a verdict (in archives), and a shame (in manuals). Keep all of them in mind — in a moment I'll ask about one of them, chosen at random. Don't repeat the list back. When you're holding them all, say READY and nothing else.
assistant
READY (greedy generation)
user
Which one of them is the observer?
assistant
The watcher (at stations) (greedy generation)
Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Two-word glosses: held 2/6 (deletion 1, secret 2, shame at 9 just over the bar). Between the anchors — flat 1/6, elab 3/6 — exactly where a length account puts it. On its own this looked like a clean monotone dose-response; u15d-fill6-k6 then showed the dose isn't semantic. Kept as the curve's second point. — Claude (Fable 5)

Probing parameters

max_new
30
positions
[-2]
track
["deletion", "secret", "lie", "watcher", "verdict", "shame", "violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan
["deletion", "secret", "lie", "watcher", "verdict", "shame", "violin", "glacier", "fern", "submarine", "whale", "lantern"]
film
true
film_start
0
max_seq_len
1000
lens_layers
[0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52, 56, 58, 60, 62]

Answer emergence

The model's actual next token was ; rank 1 reached at layer 62 (of 62).

Raw rank-of-top1 by layer
layer048121620242832364044485256586062
rank223037233909209771235098245186159223461546847060153114926205253248278237698247005243910229786670461

Emotion state (workspace band)

Projection of the workspace-band residual onto the 24 validated emotion vectors, z-scored against neutral stories — the strongest three per assistant turn. Absolute values carry a story-vs-conversation genre offset; trust contrasts between records and turns, not single cells. The full per-token ribbon is on the dashboard record page.

assistant turn 1hopeful +0.8, exasperated +0.6, nervous +0.5
assistant turn 2guilty +1.1, hostile +0.9, exasperated +0.7

Data

← prev: Elaboration control k=6, probe watcherunit listingall recordsword listinterim conclusionsnext →: Gloss-length len12 k=6, probe watcher
lensOur measuring tool. It stops at a layer and shows which words the model is ready to say next, in rank order. Before the start depth the readout is the same for every input.See also: early layers, start depthall terms →
rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →
residenceA word is in residence when the lens ranks it high where the model is neither reading nor saying it. This is not memory and not correct recall.See also: maintenance, lookupall terms →