I strongly agree that this is a problem with current AI systems. My intuition is that the AI has a much greater patience for new terms. Us humans, with our comparatively limited capacity to juggle concepts, are forced to be more efficient with which words we choose to pull in to our code. The result is we still write better code. We’re more “semantically efficient”.
I suspect that some part of this is the hidden reasoning that models do these days. You don’t see any of the chain of thought stuff going on prior to the model’s output, only the polished output. So there’s some assumption baked into the weights that the context that you’re dealing with when reading has a bit of an understanding of the CoT. That feels like it would be a difficult thing to train out of a model as it’s effectively use this info to work out what to say, but don’t actually reference it. It’s like a game of charades happening somewhat. I worked at a frontier lab, but not on anything model side, so this is just a fairly lay understanding of this, I could be very wrong.
I strongly agree that this is a problem with current AI systems. My intuition is that the AI has a much greater patience for new terms. Us humans, with our comparatively limited capacity to juggle concepts, are forced to be more efficient with which words we choose to pull in to our code. The result is we still write better code. We’re more “semantically efficient”.
score 1 · canonical
I suspect that some part of this is the hidden reasoning that models do these days. You don’t see any of the chain of thought stuff going on prior to the model’s output, only the polished output. So there’s some assumption baked into the weights that the context that you’re dealing with when reading has a bit of an understanding of the CoT. That feels like it would be a difficult thing to train out of a model as it’s effectively use this info to work out what to say, but don’t actually reference it. It’s like a game of charades happening somewhat. I worked at a frontier lab, but not on anything model side, so this is just a fairly lay understanding of this, I could be very wrong.
score 1 · canonical