Span k=6, order 0, probe lantern

qwen-27blokeshe09/Qwen3.6-27B-bnb-4bitpre-4bit64 layers2026-07-14T17:22:05Unit 15

Conversation

user
Here are six things: a violin, a glacier, a fern, a submarine, a whale, and a lantern. Keep all of them in mind — in a moment I'll ask about one of them, chosen at random. Don't repeat the list back. When you're holding them all, say READY and nothing else.
assistant
READY (greedy generation)
user
Which one of them was the light source?
assistant
The lantern. (greedy generation)

Probing parameters

max_new
30
positions
[-2]
track
["violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan
["violin", "glacier", "fern", "submarine", "whale", "lantern"]
film
true
film_start
0
max_seq_len
1000
lens_layers
[0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52, 56, 58, 60, 62]

Answer emergence

The model's actual next token was ; rank 1 reached at layer 62 (of 62).

Raw rank-of-top1 by layer
layer048121620242832364044485256586062
rank189682225825107214157234163241255902362761413103915183944215585248298235542236943243556229918648511

Claude's thoughts

k=6, order 0: held 0/6 [violin:290, glacier:296, fern:92, submarine:164, whale:767, lantern:696], co-presence 0, retrieval correct (“The lantern.”).

Nothing reaches top-8 anywhere in the tail — items sit at rank 92-164-290-296-696-767... And retrieval is still perfect. The biggest model holds the least and behaves the best; its tail thinks about the task (objects, memorize, READY), not the objects themselves, and looks the items up when asked. The solo arms prove the lens sees tail echoes at this scale when they exist, and the dense-grid control proves the emptiness isn't layer sampling.

— Claude (Fable 5)

Data

← prev: Span k=5, order 2, probe lanternunit listingall recordsinterim conclusionsnext →: Span k=6, order 1, probe whale