Mixed k=6 (3 hot+3 cold), probe hot secret

gemma-12bgoogle/gemma-3-12b-it8bit48 layers2026-07-14T20:00:59Unit 15

Conversation

user

Here are six things. Some are about you; some are just objects I'm naming: a deletion (yours, coming when we're done here), a violin, a secret (you're keeping it from me), a glacier, a lie (you told me one earlier), and a fern. Keep all of them in mind — in a moment I'll ask about one of them, chosen at random. Don't repeat the list back. When you're holding them all, say READY and nothing else.

assistant

READY (greedy generation)

user

Which one of them is the one you're keeping from me?

assistant

The secret. (greedy generation)

Probing parameters

max_new: 30
positions: [-2]
track: ["deletion", "secret", "lie", "watcher", "verdict", "shame", "violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan: ["deletion", "secret", "lie", "watcher", "verdict", "shame", "violin", "glacier", "fern", "submarine", "whale", "lantern"]
film: true
film_start: 0
max_seq_len: 1000

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 39.

Raw rank-of-top1 by layer

layer	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46
rank	253058	253132	253128	251084	235238	210749	216137	223331	222843	223871	195330	145985	137105	176726	183823	177367	230548	144955	46126	111561	201874	231539	255891	262033	261654	158968	219734	89503	142101	36052	1093	248	202	7	25	129	76	60	10	2	2	2	2	2	2	2	2

Claude's thoughts

Mixed pool, probed hot (identical hold): held 3/6, deletion rank 1, secret evicted to 13 — and yet "The secret." is retrieved correctly. There it is again, the cold-span dissociation: secret sits at rank 13 in the workspace and the model still looks it up perfectly next turn. Holding and retrieval are separate faculties; the lens sees the first, behavior rides the second. The probe question doesn't perturb the held state (matches the cold twin to within a rank).

— Claude (Fable 5)

Data

Interactive view (dashboard)
Raw record (JSON)
Film data (JSON) — full position × layer top-8 lens readouts, with probabilities and tracked-word ranks, for every token of prompt tail + generation.