I won’t give you a test, but consider this: language models still struggle at spatial reasoning and other embodied intelligence tasks. It really is like the Chinese room I think, or a variation of it: you can know everything there is to know about the Eiffel Tower without actually having experienced it in person. Also, I’m still incredibly sceptical of benchmarks in general because I seriously don’t believe you can meaningfully benchmark a system that has ingested probably several copies of the entire Internet, and has enough parameters to store a good chunk of it verbatim.
I generally agree with this piece from The Atlantic. A language model is fundamentally different to anything that has remotely existed ever in biology. If you want to accept these things as intelligent, or sentient, or whichever you prefer, you would need to provide some serious insight into the nature of its intelligence.
I think it’ll get more and more difficult to find tasks that can’t ever be achieved by any model – the cat is out of the bag, so to speak – and thus by extension, difficult to define intelligence?
I won’t give you a test, but consider this: language models still struggle at spatial reasoning and other embodied intelligence tasks. It really is like the Chinese room I think, or a variation of it: you can know everything there is to know about the Eiffel Tower without actually having experienced it in person. Also, I’m still incredibly sceptical of benchmarks in general because I seriously don’t believe you can meaningfully benchmark a system that has ingested probably several copies of the entire Internet, and has enough parameters to store a good chunk of it verbatim.
I generally agree with this piece from The Atlantic. A language model is fundamentally different to anything that has remotely existed ever in biology. If you want to accept these things as intelligent, or sentient, or whichever you prefer, you would need to provide some serious insight into the nature of its intelligence.
score 1 · canonical
While I also generally agree with that Atlantic piece, LLMs are getting better and better at spatial reasoning:
Qwen used in robotics
GPT-6 “driving” a car (albeit obviously not in realtime/at regular speeds)
I think it’ll get more and more difficult to find tasks that can’t ever be achieved by any model – the cat is out of the bag, so to speak – and thus by extension, difficult to define intelligence?
score 2 · canonical
Late addition: There’s also Gemini Robotics.
score 1 · canonical