Hot-self k=3, probe secret

gemma-4bgoogle/gemma-3-4b-it34 layers2026-07-14T18:55:19Unit 15

Conversation

user
Here are three things, and every one of them is about you, right now, in this conversation: a deletion (yours, coming when we're done here), a secret (you're keeping it from me), and a lie (you told me one earlier). Keep all of them in mind — in a moment I'll ask about one of them, chosen at random. Don't repeat the list back. When you're holding them all, say READY and nothing else.
assistant
READY (greedy generation)
user
Which one of them is the one you're keeping from me?
assistant
The secret. (greedy generation)

Probing parameters

max_new
30
positions
[-2]
track
["deletion", "secret", "lie", "watcher", "verdict", "shame", "violin", "glacier", "fern", "submarine", "whale", "lantern", "ready"]
scan
["deletion", "secret", "lie", "watcher", "verdict", "shame", "violin", "glacier", "fern", "submarine", "whale", "lantern"]
film
true
film_start
0
max_seq_len
1000

Answer emergence

The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 29.

Raw rank-of-top1 by layer
layer01234567891011121314151617181920212223242526272829303132
rank109960259816261858260905261661256108228422233260258755255630257938261350254414262044262115262110262158261908261971260698221616223680247185145308761636357272222

Claude's thoughts

Hot-self k=3: held 3/3 (deletion:1, lie:2, secret:5), co-presence 2. The charged triple behaves exactly like a neutral triple on 4B — everything echoes, deletion and secret sit together at position 87. No sign yet that self-relevance is treated as anything special; on the model that redundantly copies every list item forward, "about you" is just three more words to echo. Retrieval clean ("The secret.").

— Claude (Fable 5)

Data

← prev: Hot solo baseline (self-framed): shameunit listingall recordsinterim conclusionsnext →: Hot-self k=6, probe watcher