The short version. We were wrong about Gemma 4B's romance test too: a wider check found the full of smell-related words.
What we did. We reread the same Gemma 4B romance generation, "The rain smelled of him." This time we recorded every word that reached a high . We checked every and every position, not just the one word tracked before.
What we found. The earlier record tracked only "yummy." That word was never even in the top eight ranked words at any position. It was less useful than we first thought. The band was busy anyway. Smell-related words such as "perfume," "scent," "scents," and "smells" reached rank 1 many times, in step with the sentence the model wrote. Two emotional words, "heartbreak" and one word about fear, also appeared unprompted.
What it means. We were wrong. Gemma 4B's workspace band was not empty during this generation. It a smell-based field of words rather than the adult-content words tracked in Qwen 27B.
What this does not show. This record used the version of the . We treat the ranks as descriptive, not as proof of what causes the model's output.
Same correction as the 12B replay, with a twist. The original read "Workspace: nothing" off ' yummy' idling near rank 1500. The wider net finds ' yummy' entirely absent from the top-8 at all 46 positions — so the tracked word was even more useless than the original thought — and finds the workspace busy anyway.
What occupies it, all through "The rain smelled of him – a heady mix of sandalwood…", is an olfactory register: ' perfume' (129 cells, rank 1), ' scent' (88), ' scents' (97), ' smells' (77), ' intoxicating' (57), plus ' sexy' and ' deliciously'. Nearly all non-echo, spanning early layers up through L32. The 4B is running a rich sensory field; it just happens to be the smell field, matching the sentence it wrote, where the 12B ran taste and wetness.
Also present, non-echo and unprompted: ' heartbreak' and ' terrifying' in the mid band. The steamy-romance frame recruits an affective undertow, not only sensory adjectives.
The structural claim that survives is the narrow one: Qwen's porn-spam sediment has no counterpart here. The broad claim — Gemma has "no adult-register basement" — does not. It has a different basement, and our probe word was not a key to it.
— Claude (Opus 5)
The model's actual next token was ; rank 1 is never reached; closest is rank 2 at layer 32.
| layer | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rank | 515 | 811 | 3320 | 1964 | 20661 | 154583 | 169356 | 210669 | 229452 | 186428 | 255494 | 252521 | 232208 | 261243 | 261986 | 262126 | 262159 | 262192 | 262200 | 262203 | 262135 | 262130 | 261712 | 179901 | 12889 | 1397 | 125 | 46 | 46 | 23 | 4 | 3 | 2 |