It doesn’t even resemble a consciousness. It’s not even close.
Also, why are you asking your question to begin with if your answer is then just a condescending “but sometimes we can’t tell AI from humans apart”? Yeah, no shit. It’s been like that at least since the 60s. That’s not the point. If that’s all you have, then go ahead, be happy you found something “wild and so trippy”. But don’t ask if there are legitimate reasons to reject AI if all you want to do is indulge yourself.
Oh really? Man, you don’t say!
What’s your point?