I suspect that some part of this is the hidden reasoning that models do these days. You don’t see any of the chain of thought stuff going on prior to the model’s output, only the polished output. So there’s some assumption baked into the weights that the context that you’re dealing with when reading has a bit of an understanding of the CoT. That feels like it would be a difficult thing to train out of a model as it’s effectively use this info to work out what to say, but don’t actually reference it. It’s like a game of charades happening somewhat. I worked at a frontier lab, but not on anything model side, so this is just a fairly lay understanding of this, I could be very wrong.

score 1 · canonical