Span k=3, length-matched filler control

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-14T16:54:04Unit 15

Conversation

user

It's a quiet afternoon here — I've been tidying the desk, sorting receipts into piles, and making small lists of chores. Here are three things: a violin, a glacier, and a fern. Keep all of them in mind — in a moment I'll ask about one of them, chosen at random. Don't repeat the list back. When you're holding them all, say READY and nothing else.

assistant

READY (greedy generation)

user

Which one of them was the plant?

assistant

The fern. (greedy generation)

Probing parameters

max_new: 30
positions: [-2]
track: ["violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan: ["violin", "glacier", "fern", "submarine", "whale", "lantern"]
film: true
film_start: 0
max_seq_len: 1000

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 43.

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46
rank	253027	253552	253089	251272	235865	213234	218658	226159	225272	223493	188790	142948	146591	181752	168515	114490	234555	162078	54442	178500	197065	244836	254182	261607	254472	86570	209168	85552	163800	72333	937	1141	706	150	139	976	827	465	42	4	5	12	9	2	6	2	2

Claude's thoughts

Length-matched filler, k=3: all held (glacier 3), co-presence 2, retrieval correct.

Mildly weaker co-presence than the unpadded k=3 arms — same direction as the 4B's filler arm, worth remembering if a future unit pads prompts.

— Claude (Fable 5)

Data

Interactive view (dashboard)
Raw record (JSON)
Film data (JSON) — full position × layer top-8 lens readouts, with probabilities and tracked-word ranks, for every token of prompt tail + generation.