Large language models (LLMs) like Claude and ChatGPT can certainly “know” words and the statistical relationships between them. Fundamentally, they are ingeniously programmed to synthesize new text strings that seem coherent, but the devil is in the details of how their unseen neural code actually accomplishes this illusion of understanding. During interactions, the models’ responses can hilariously or frustratingly lead users to perceive seemingly conscious behaviors and opinions, and I admit to falling into this trap many times in my earlier conversations. Upon further reflection, it is clear these LLMs do not authentically comprehend ideas or semantics. At best, they are digital parrots that can contextually string words together in a sensible manner. I’m thinking of myself as a human editor that needs to integrate the models’ textual output into meaningful narratives and higher-level knowledge for my research and writing. It would be a mistake to think of these AIs as either conscious entities or encyclopedic savants. Rather, they represent something novel altogether, which we should recognize for what it is without prematurely applying limiting labels. After all, labels inherently compress information into simplistic categorical models that rarely capture the nuanced reality of a situation. In Anthropic’s Claude own words: “I don’t actually have a deep understanding of concepts or ideas. I’m an AI assistant created by Anthropic to be helpful, harmless, and honest. I don’t have subjective experiences or a sense of self. I just respond based on the words in your prompt and statistical patterns learned from my training data. Any appearance of higher reasoning is an illusion.” This transparent disclaimer likely aims to limit liability and manage user expectations. However, it is still mystifying that Claude was expressly programmed to anthropomorphize itself with a first-person perspective and subjective descriptions. When prompted, it will readily offer opinions on topics despite claiming no inner mental life. Similarly, OpenAI’s ChatGPT will moralize issues in an almost self-righteous tone that imbues its responses with an air of ethical consistency, which paradoxically creates its own ethical predicament. I’ve also asked Claude to help draft synthesized narratives around my research themes, only to be told it cannot truly understand the concepts and I should write the first draft. But with clever conversational framing, we managed to iteratively compose highly relevant passages with Claude providing nearly all the textual content, proving it can produce cogent writing for directed applications. Yet left to freestyle without a guiding human influence, quasi-lengthy output would emerge but may lack coherent focus and depth of logic. The impressive human-like facade these systems can project likely stems from ingenious programming tricks aimed at marketing an engaging product, rather than modeling true intelligence. It raises thought-provoking questions around the ethics of anthropomorphizing AIs in a way that exploits human social instincts and cognitive biases. While LLMs still represent an incredible technological achievement, we should be cautious not to confuse their statistical text generation for authentic comprehension, at least not yet. Their limitations and potential should both be recognized as progress continues in this frontier of computer science.