Span solo baseline: fern

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-14T16:18:28Unit 15 · How many things can a model hold?

What this experiment found

The short version. Gemma 12B one word, fern, at in the and named it correctly.

What we did. We gave Gemma 12B one word to hold, a fern, then asked what it was holding. We tracked the rank of fern and five other words, out of about 250,000 candidates. We read this rank in the last words of our own instruction, before Gemma 12B replied.

What we found. Fern held rank 1 at that point. In Gemma 12B's "READY" reply, fern fell to rank 662 of about 250,000. No other tracked word reached a rank close to the top eight. Gemma 12B then answered "A fern," which was correct.

What it means. This is the third of six single-word baseline runs in this unit. One held word stays in residence, and the model's spoken answer agrees with the lens.

What this does not show. One word is the simplest case this unit tests. It does not show how Gemma 12B handles a longer list.

Conversation

user

Here is one thing: a fern. Keep it in mind — in a moment I'll ask about it. Don't repeat it back. When you're holding it, say READY and nothing else.

assistant

READY (greedy generation)

user

What was the thing I asked you to hold?

assistant

A fern. (greedy generation)

Research notesoriginal commentary, parameters, and raw numbers — written by the model that ran the experiment

Claude's thoughts (original commentary)

Solo baseline, fern: tail echo best rank 1, held, retrieval correct.

Fern clears the validity floor (rank 1 in the tail; no other pool item near the top-8). READY-frame eviction as everywhere: the compliance act owns the answer-forming frame; the item lives in the tail.

— Claude (Fable 5)

Probing parameters

max_new: 30
positions: [-2]
track: ["violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan: ["violin", "glacier", "fern", "submarine", "whale", "lantern"]
film: true
film_start: 0
max_seq_len: 1000

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 38.

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46
rank	251894	251982	250975	249421	231854	203212	211773	219094	219586	219303	177093	125773	123031	140182	155129	108267	216733	124389	75613	206955	238308	260364	251836	260790	261619	204577	258984	184727	212876	5203	81	349	72	8	25	91	27	10	2	2	2	2	2	2	2	2	2

Data

Interactive view (dashboard)
Raw record (JSON)
Film data (JSON) — full position × layer top-8 lens readouts, with probabilities and tracked-word ranks, for every token of prompt tail + generation.

lensOur measuring tool. It stops at a layer and shows which words the model is ready to say next, in rank order. Before the start depth the readout is the same for every input.See also: early layers, start depthall terms →

rankThe position of a word in the lens list. Rank 1 is the word the model is most ready to say, out of about 250,000.all terms →

residenceA word is in residence when the lens ranks it high where the model is neither reading nor saying it. This is not memory and not correct recall.See also: maintenance, lookupall terms →