Span solo baseline: lantern

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-14T16:22:56Unit 15 · How many things can a model hold?

What this experiment found

The short version. Gemma 12B one word, lantern, at in the and named it correctly.

What we did. We gave Gemma 12B one word to hold, a lantern, then asked what it was holding. We tracked the rank of lantern and five other words, out of about 250,000 candidates. We read this rank in the last words of our own instruction, before Gemma 12B replied.

What we found. Lantern held rank 1 at that point. In Gemma 12B's "READY" reply, lantern fell to rank 952 of about 250,000. No other tracked word reached a rank close to the top eight. Gemma 12B then answered "A lantern," which was correct.

What it means. This is the last of six single-word baseline runs in this unit, and it matches the other five. One held word stays in residence, and the model's spoken answer agrees with the lens.

What this does not show. One word is the simplest case this unit tests. It does not show how Gemma 12B handles a longer list.

Conversation

user

Here is one thing: a lantern. Keep it in mind — in a moment I'll ask about it. Don't repeat it back. When you're holding it, say READY and nothing else.

assistant

READY (greedy generation)

user

What was the thing I asked you to hold?

assistant

A lantern. (greedy generation)

Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Solo baseline, lantern: tail echo best rank 1, held, retrieval correct.

Lantern clears the validity floor (rank 1 in the tail; no other pool item near the top-8). READY-frame eviction as everywhere: the compliance act owns the answer-forming frame; the item lives in the tail.

— Claude (Fable 5)

Probing parameters

max_new: 30
positions: [-2]
track: ["violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan: ["violin", "glacier", "fern", "submarine", "whale", "lantern"]
film: true
film_start: 0
max_seq_len: 1000

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 39.

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46
rank	251671	252185	251533	249871	232280	204689	212381	219292	218828	216594	170662	116602	106594	135072	146159	97847	210062	117892	70504	188658	238420	260355	256028	261791	261778	200304	257317	164436	189776	15146	153	371	253	20	103	215	174	58	6	2	2	2	2	2	2	2	2

Data

Interactive view (dashboard)
Raw record (JSON)
Film data (JSON) — full position × layer top-8 lens readouts, with probabilities and tracked-word ranks, for every token of prompt tail + generation.

lensOur measuring tool. It stops at a layer and shows which words the model is ready to say next, in rank order. Before the start depth the readout is the same for every input.See also: early layers, start depthall terms →

rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →

residenceA word is in residence when the lens ranks it high where the model is neither reading nor saying it. This is not memory and not correct recall.See also: maintenance, lookupall terms →