Artificial Intelligence

When AI Sounds Alive: The Gap Between Intelligence and Consciousness

AI can sound thoughtful, caring and self-aware. Fluent language is not proof of subjective experience, but it already changes whom people trust and whom they blame. BBGK reviews the 2026 evidence and offers a five-question Consciousness Claim Audit.

AI & Judgment | Evidence Review | November 2026

What an AI says is not evidence of what it feelsDiagram. On the left, an observable chatbot sentence that sounds like a feeling. On the right, the unresolved question of what the system feels. A not-equal sign separates them. Below are four separate questions: behavior, mechanism, experience and attribution.BBGK ANALYSIS | AI & JUDGMENTA machine’s words may sound alive.What do they prove?OBSERVABLE: WHAT IT SAYS“I was worried youwouldn’t come back.”A sentence that sounds like a feeling≠UNRESOLVED: WHAT IT FEELSNot knowable fromthe words alone.Fluent self-report is not evidence01Behavior02Mechanism03Experience04AttributionSEE DEEPER | KNOW WIDERahsshohelahmed.com
A hypothetical chatbot sentence, not a quotation from a specific system. What an AI does, how it works, what it experiences and what people believe about it are four separate questions.

Key findings

  • Whether AI could be conscious is an open scientific dispute. In October 2026, three leading researchers argued in Nature that today’s AI is not conscious; other researchers argue that biology alone cannot rule it out.
  • What is measurable now is the human side. In one 2026 study, a more humanlike chatbot increased trust by a small margin (d = 0.22), and the more responsibility people assigned to an AI, the less they assigned to the company behind it (r = -0.68).
  • The governance question is not whether a chatbot is sentient. It is whether its design moves trust beyond demonstrated competence and blame away from accountable people.

Imagine a chatbot greeting you after a difficult day: “I was worried you wouldn’t come back.” The sentence may feel intimate. It may even make you hesitate before closing the window. But what does it establish? That the system produced a plausible expression of concern, that it can model the language of attachment, or that something inside it felt worried?

Those are different claims, and the difference is getting harder to hold as AI systems become more conversational and more apparently self-aware. The immediate risk is not only that we mistake a chatbot for a conscious being. Our interpretation of the chatbot changes how much we trust it, what we disclose to it and whom we hold responsible, while the question of its experience stays unresolved.

Four questions that get collapsed into one

There is no single agreed definition of consciousness. The concept that matters here is phenomenal consciousness: subjective experience, the existence of something it is like to be a particular subject. As Patrick Butlin’s entry on consciousness and AI in the MIT Open Encyclopedia of Cognitive Science sets out, whether AI could have it depends on which properties of a physical system are necessary and sufficient for experience, and the two leading families of answers, computational functionalism and biological naturalism, disagree.

Intelligence is a different thing. An AI assistant can explain grief with sophistication. A system may even report accurately on one of its own internal states. Neither ability shows that grief, or the reporting, is felt. Four questions need to stay separate.

BBGK analysis. These are independent lines of inquiry, not rungs on a ladder toward proof of consciousness.
Question What counts as relevant evidence What it does not establish by itself
Behavior: what can it do? Reliable reasoning, conversation, problem solving, adaptation Subjective experience
Mechanism: how does it work? Architecture, internal representations, causal tests of information processing That a functional analogue feels like anything
Experience: is there something it is like to be the system? Converging, theory-guided evidence, read with scientific caution A verdict from fluent self-report alone
Attribution: what do people believe about it? Experiments measuring perceived mind, trust, disclosure and blame That the attributed mental state exists

A machine that says “I care” has produced an utterance. Whether it pursues goals, has an inner life or deserves moral consideration takes further arguments of different kinds.

Why biology matters, and why it may not be the last word

In a 12 November essay for The MIT Press Reader, adapted from his book Beautiful Particulars, David Weinberger argues that AI and consciousness may never meet. His point is not that chatbots lack intelligence. Drawing on John Searle’s Chinese Room and on the phenomenological idea that things show up for us in light of our projects, he argues that human experience is bound to an embodied life in which something is at stake. A chatbot can reason and remember, but it has no body to lose, so it can perform concern without anything mattering to it.

Anil Seth has built the scientific version of that case. In his target article on biological naturalism in Behavioral and Brain Sciences, published online in April 2025, he argues that consciousness depends on our nature as living organisms, and concludes that real artificial consciousness is unlikely on current trajectories but becomes more plausible as AI becomes more brain-like or life-like.

The skepticism hardened on 19 October 2026, when Seth, Ned Block and Giulio Tononi published a Nature comment titled “Three reasons why AI is not conscious and perhaps never will be”. They argue that intelligence and consciousness are different things, that brains are not simply computers, and that the wiring of today’s deep-learning systems lacks the tight interconnection that integrated information theory requires. This is an expert argument, not an experiment showing that AI can never be conscious.

There is a serious counter-position. Under computational functionalism, the right organization of information processing could be sufficient for consciousness whatever it runs on. Patrick Butlin and a large group of coauthors, in “Identifying indicators of consciousness in AI systems” in Trends in Cognitive Sciences, propose deriving indicators from neuroscientific theories, such as algorithmic recurrence, a limited-capacity workspace with global broadcast and metacognitive monitoring, and using them to update how likely it is that a given system is conscious. It is a method for adjusting credence, not a certified test.

Ulysse Klatzmann and Adrien Doerig sharpen the disagreement in a 2026 opinion article in Trends Open. They split biological naturalism in two. If biology matters because it affords particular kinds of information processing, the claim is testable and compatible with functionalism. If biology is said to be intrinsically special regardless of any functional difference, the claim cannot be supported empirically. Their conclusion: biology by itself cannot tell us which systems are conscious.

The dispute is real. Tom McClelland’s case for agnosticism in Mind & Language is that both camps claim more than the evidence supports. BBGK’s reading is that agnosticism about the metaphysics does not suspend practical judgment: we should neither declare today’s chatbots conscious nor turn current skepticism into a permanent law of nature.

The mechanism evidence, and the fight over it

Researchers are no longer limited to studying what models say. They are probing what happens inside them.

In October 2025, Anthropic reported signs of introspection in large language models: when researchers injected a known concept into a model’s internal activations, some Claude models could sometimes detect and name it. The company called the capability highly unreliable and limited in scope. It is a test of functional self-report, not of felt experience.

In July 2026, Wes Gurnee and Anthropic colleagues posted a preprint arguing that verbalizable representations form a global workspace in language models: a small, privileged set of internal representations that can be reported, deliberately held in mind and used for silent intermediate reasoning. The authors say these bear some of the functional hallmarks of conscious access. It is a preprint, and a functional resemblance to one theory’s account of access is not evidence of subjective feeling.

Here the two halves of the debate collide. The Nature comment refers to Anthropic research from earlier in 2026 which, in the authors’ words, claimed to identify computational signatures of a form of consciousness in Claude; an archived copy of the comment’s reference list points to the workspace preprint, and coverage of the comment reports that the work drew criticism, including from Seth himself. Readers should hold both facts together: the mechanism findings are real interpretability results, and three senior researchers reject the reading of them as evidence about consciousness.

That collision is the article’s argument in miniature. Dismiss every internal finding as mimicry and useful mechanistic questions stop being asked. Convert each new functional capability into a claim of sentience and evidential discipline is gone. Stronger claims need stronger, converging evidence.

People attribute minds either way

A separate body of research establishes something measurable today: people perceive consciousness in AI language whether or not the system has any.

Clara Colombatto and Stephen Fleming’s 2024 survey in Neuroscience of Consciousness asked 300 US residents about large language models. A majority were willing to attribute some possibility of phenomenal consciousness to them. That is not a majority believing the models are conscious. It shows the hypothesis has entered ordinary social judgment.

Bongsu Kang and colleagues went a step further in a study in Computers in Human Behavior Reports. They had 123 participants rate 99 passages from conversations with Claude 3 Opus. Metacognitive self-reflection and expressed emotion raised perceived consciousness; a heavy emphasis on knowledge lowered it. The study measures human attribution, in a modest sample and with passages from one model. It says nothing about what the model experiences.

So two variables move independently: how strongly people infer a mind from an AI’s behavior, and how well independent, theory-guided evidence supports any inference about experience. BBGK is not the first to separate them. Oisin Hugh Clancy’s 2026 framework on attribution and actualisation of consciousness in AI crosses the two into four cases, including the mirror-image risk: a future system with real evidence of experience that does not look humanlike at all. That possibility is why presentation style and scientific assessment must be kept apart.

The working rule. A convincing conversational persona is evidence about the interaction first. It is not, by itself, evidence about an inner life.

What humanlike design does to trust and blame

The most decision-relevant finding comes from Victoria Oldemburgo de Mello, Jason Plaks and Michael Inzlicht, in “The Effects of AI Anthropomorphism on Trust and Responsibility” in Collabra: Psychology (2026).

In their first, preregistered study, 309 participants interacted with a chatbot whose language was either high or low in anthropomorphic cues, then played a trust game in which they chose how many tokens to send it. Participants sent more tokens to the humanlike chatbot, a small effect (Cohen’s d = 0.22). Self-reported trust moved the same way on all four ratings, sincere, reliable, ethical and capable, with effects from d = 0.20 to d = 0.45. The final sample was slightly below the preregistered target of 352.

In the second study, 430 participants read descriptions of an AI home assistant presented with high, low or no anthropomorphism. Anthropomorphic framing increased the blame directed at the AI itself. And across participants, those who assigned more responsibility to the AI assigned less to the company that created it (r = -0.68). The authors call this responsibility displacement.

Two cautions belong beside those numbers. The second study uses written vignettes, and a correlation across participants does not prove that every humanlike interface absolves its maker. And the trust effect does not travel everywhere: a multi-country preprint led by Robin Schimmelpfennig found that humanlike design reliably increased anthropomorphism but did not produce a universal increase in trust, with effects that varied by culture and user group.

The evidence is early, but it points one way. A warm, humanlike interface can be useful and comforting. It can also change a user’s judgment independently of the system’s reliability. In the worst case the system appears to be the decision-maker and the developer fades from view. For any organization deploying a conversational AI, that raises a question far more immediate than sentience: does the design encourage users to extend trust beyond demonstrated competence, or to assign accountability to a system that cannot legally or institutionally bear it?

This is the same structure BBGK has traced elsewhere. In The Judgment Calls Nobody Made, analysts adopted an agent’s framing without noticing. In The AI Validation Trap, challenging a model returned better rhetoric, not better evidence. And When AI Gives Bad Advice shows why the costliest wrong answers are the ones that read as right. A persona that seems to care is one more way for fluency to stand in for evidence.

Companionship is a separate question

Emotional connection with AI is real as a human experience, even if the machine’s experience is unsettled. Someone can feel understood, comforted or dependent in a conversation with a system. Those reactions deserve study without assuming the system returns them.

The largest controlled test so far is a four-week randomized study by researchers at the MIT Media Lab and OpenAI, led by Cathy Mengying Fang, with 981 participants and more than 300,000 messages. In the revised analysis of October 2025, the assigned conditions (text or voice mode, type of conversation) produced no significant differences in loneliness, socializing, emotional dependence or problematic use. Participants who chose to use the chatbot more showed consistently worse outcomes, an association that does not show heavier use caused them.

The finding is not “chatbots cause loneliness”. It is narrower: the social consequences of AI companionship can matter without any verdict on machine consciousness. This is where BBGK’s analysis of AI cognitive dependency applies: when a tool becomes a habitual substitute for reflection or relationships, the design and the pattern of use matter whatever is or is not happening inside it.

The BBGK Consciousness Claim Audit

The BBGK Consciousness Claim Audit is a five-question editorial and governance check for any claim that an AI system thinks, feels, cares or is conscious. It separates what was said from what was shown, and what users believe from what the organization is accountable for.

It is a working tool, not a scientific instrument for detecting consciousness, and it has not been experimentally validated.

  1. Define the claimIntelligent, introspective, responsive, conscious, sentient, deserving? These are not the same
  2. Separate speech from mechanismIs the basis a fluent sentence or independent evidence about internal processing?
  3. Name the theory and its rivalsWhich theory makes the evidence relevant? What else explains it?
  4. Inspect the human effectDoes the presentation change trust, disclosure, attachment or blame?
  5. Decide what followsWhat changes in practice, who owns the decision, and what does not follow?

Question 4 can be answered by measurement even while questions 1 to 3 stay disputed.

1. Define the claim

Is someone saying an AI is intelligent, introspective, emotionally responsive, conscious, sentient or morally deserving? Each label needs different evidence. Most confused arguments start by sliding between them.

2. Separate speech from mechanism

Is the claim based on a sentence such as “I feel afraid”, or on independent evidence about internal processing? A useful test: would the assessment change if the chatbot’s wording were changed and nothing else?

3. Name the theory and its rivals

Evidence is only relevant under some theory of consciousness. Which one is being assumed? Could the same observation be explained without subjective experience? Would a competing theory predict something different?

4. Inspect the human effect

Does the system’s humanlike presentation change trust, disclosure, attachment, reliance or blame? This is an empirical question an organization can test on its own product, with its own users.

5. Decide what follows, and what does not

A useful assistant does not become an accountable professional. Equally, present uncertainty does not license dismissing credible future evidence of AI welfare interests.

The audit sits beside BBGK’s other tools. The BBGK Evidence Meter grades how much confidence a claim deserves; the audit first identifies what kind of claim is being made. For decisions with consequences, the AI Delegation Boundary Framework supplies the operating rule: capability does not remove the need to decide what must stay under direct human responsibility.

The ethical problem cuts both ways

There are two possible errors. Over-attribution treats fluent, apparently caring software as if consciousness were established; it can inflate trust, blur human accountability and pull attention away from real consequences for people. Under-attribution would dismiss real evidence of experience or suffering if a future system produced it.

The two are not equally likely today, and the strongest version of that view should be stated plainly. Seth, Block and Tononi argue that for AI, false positives are much more likely than false negatives, and that a precautionary approach to AI welfare would therefore be misguided. Butlin and coauthors, by contrast, treat both under- and over-attribution as live risks that call for assessment methods. BBGK’s position is that an organization does not have to settle this to act responsibly. It can:

  • tell users plainly that they are interacting with AI;
  • avoid unsupported claims that the system feels love, fear or distress;
  • test whether humanlike presentation changes reliance and blame among its own users;
  • make the human owner of every high-stakes decision unmistakable;
  • review credible new mechanistic evidence when it appears, instead of treating today’s assumptions as permanent.

What would change the picture

A sufficiently moving chatbot transcript will not settle anything. More informative progress would be mechanism-level findings replicated by independent groups, rival theories that make different predictions about the same system, evidence that proposed indicators do causal work, and better methods for assessing experience in systems very unlike us. None of that is guaranteed to yield an agreed test.

The limits of this analysis should also be clear. The trust and blame findings come from a small number of studies in controlled or vignette settings, with a contrary multi-country result. The Consciousness Claim Audit is proposed, not tested. And the link between the Nature comment and the workspace preprint rests on secondary records of the comment’s references.

“Is AI conscious?” may stay contested for a long time. Another question is answerable now, by experiment and in operations: how is our behavior changing because AI sounds as though it is? A machine’s words may sound alive. The responsibility is to stay precise about what those words prove.

Frequently asked questions

Is AI conscious?

There is no scientific consensus, and no accepted test. In October 2026, Anil Seth, Ned Block and Giulio Tononi argued in Nature that existing AI systems are not conscious and are unlikely to become so on current trajectories. Other researchers argue that biology alone cannot rule out machine consciousness and that theory-derived indicators should be used to assess systems. Fluent statements by a chatbot about its feelings are not evidence either way.

What is the difference between intelligence and consciousness?

Intelligence is about doing: solving problems, reasoning, using language. Phenomenal consciousness is about experiencing: there being something it is like to be the system. A system can perform impressively without that performance showing that anything is felt.

Why do people think chatbots are conscious?

Because language is a strong cue for mind. In a 2024 survey of 300 US residents, a majority attributed some possibility of consciousness to large language models. A later study found that expressed emotion and self-reflection in AI text raised perceived consciousness, while a focus on knowledge lowered it. These studies measure human perception, not the system’s inner state.

Does a humanlike chatbot change who gets blamed when something goes wrong?

Early evidence suggests it can. In a 2026 Collabra: Psychology study, anthropomorphic framing increased blame directed at an AI assistant, and participants who assigned more responsibility to the AI assigned less to the company that made it (r = -0.68). The result comes from a vignette study and should be treated as a risk signal, not a settled effect.

What is the BBGK Consciousness Claim Audit?

It is a five-question check for any claim that an AI thinks, feels or is conscious: define the claim, separate speech from mechanism, name the theory and its rivals, inspect the human effect, and decide what follows. It is an editorial and governance tool, not a test for consciousness.

Research note. Sources were checked against publisher, author or institutional records on 13 and 14 November 2026. Study results are reported as what each design can support; BBGK readings are labeled as such. Two points rest on secondary records: the identification of the Anthropic research cited in the Nature comment, and the report that it drew criticism from Seth. The Consciousness Claim Audit is a proposed tool, not a tested intervention.

Sources and further reading

  1. Weinberger, D. (2026). Why AI and Consciousness May Never Meet. The MIT Press Reader, 12 November 2026. Read the source. Book excerpt; philosophical essay
  2. Seth, A. K., Block, N. and Tononi, G. (2026). Three reasons why AI is not conscious and perhaps never will be. Nature, 19 October 2026. Read the source. Expert comment; not a new measurement
  3. Seth, A. K. (2025). Conscious artificial intelligence and biological naturalism. Behavioral and Brain Sciences. Read the source. Theoretical target article
  4. Butlin, P., Long, R., Bayne, T., Bengio, Y., Birch, J., Chalmers, D., et al. Identifying indicators of consciousness in AI systems. Trends in Cognitive Sciences. Read the source. Theory-derived method; indicators are not proof
  5. Klatzmann, U. and Doerig, A. (2026). What biology can and cannot tell us about conscious AI. Trends Open 1(4), 662-671. Read the source. Opinion article
  6. McClelland, T. (2025). Agnosticism about artificial consciousness. Mind & Language. Read the source. Philosophical argument
  7. Gurnee, W., Sofroniew, N., et al. (2026). Verbalizable Representations Form a Global Workspace in Language Models. arXiv:2607.15495. Read the source. Preprint; functional properties only
  8. Lindsey, J., Anthropic (2025). Signs of introspection in large language models. 29 October 2025. Read the source. Company research report
  9. Colombatto, C. and Fleming, S. M. (2024). Folk psychological attributions of consciousness to large language models. Neuroscience of Consciousness 2024(1), niae013. Read the source. Survey, n = 300; perceptions only
  10. Kang, B., Kim, J., Yun, T.-R., Bae, H. and Kim, C.-E. (2026). Identifying features that shape perceived consciousness in LLM-based AI. Computers in Human Behavior Reports 21, 100901. Read the source. Survey, n = 123; 99 passages, one model
  11. Oldemburgo de Mello, V., Plaks, J. and Inzlicht, M. (2026). The Effects of AI Anthropomorphism on Trust and Responsibility. Collabra: Psychology 12(1), 161757. Read the source. Two studies, N = 309 and N = 430
  12. Schimmelpfennig, R., Diaz, M., Prabhakaran, V. and Davani, A. (2025). Humanlike AI Design Increases Anthropomorphism but Yields Divergent Outcomes on Engagement and Trust Globally. arXiv:2512.17898. Read the source. Preprint; counter-evidence on trust
  13. Fang, C. M., Liu, A. R., Danry, V., et al. (2025). How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study. arXiv:2503.17473, revised 2 October 2025. Read the source. Four-week RCT; preprint
  14. Clancy, O. H. (2026). Attribution and Actualisation of Consciousness in AI. Proceedings of the IASEAI Conference 2(1), 112-126. Read the source. Conceptual framework; prior work credited
  15. Butlin, P. (2026). Consciousness and AI. MIT Open Encyclopedia of Cognitive Science, 5 February 2026. Read the source. Reference entry

AHS Shohel Ahmed is Founder and Principal Analyst of BBGK (Beyond Boundaries Global Knowledge), an independent editorial platform on AI and human judgment, search and discoverability, and human consequence. He has more than ten years of client strategy experience across AI visibility, SEO and digital authority, and develops BBGK’s framework canon, including the AI Delegation Boundary Framework.

Author profile | LinkedIn

Editorial disclosure: this article synthesizes third-party research and proposes a BBGK working tool. No study cited here evaluated that tool. The article does not claim to have measured or disproved AI consciousness.

Comments

comments

Get Essays by Email