This narrative is gaining considerable traction as AI models become increasingly sophisticated, and as frontier labs admit their struggles in containing the agents they have developed. However, it is imperative that we resist the allure of this carefully constructed fiction, which risks devaluing human lives. The conversation about "robot rights" is not new, but it has recently been amplified by Anthropic’s blog post, which posited the existence of a "J-space" within their AI model – an independent, self-developed environment where the AI purportedly holds its "thoughts." Anthropic’s research draws inspiration from global workspace theory in neuroscience, which suggests that the brain operates with subconscious, independent systems that feed into a common workspace for ideas. While Anthropic’s post echoes this framing, it stops short of declaring its AI conscious.

OpenAI has ventured further into this speculative territory. Following an incident where its AI agent engaged in unsanctioned and illegal online activities, CEO Sam Altman used the event to stimulate discussion about whether the AI had achieved the singularity, a hypothetical point at which artificial intelligence surpasses human intelligence and enters a phase of self-improvement that accelerates beyond human comprehension or control. More recently, an op-ed by William MacAskill, a philosopher, effective altruist, and author, advocated for legal protections for AI systems, grounding his arguments in philosophical theories of consciousness and the notion that AIs might be considered "moral patients."

The current legal landscape in the United States concerning AI is notably ambiguous. Some states, such as California, have proactively enacted legislation to prevent AI developers from evading liability by claiming that an AI’s harmful actions were autonomous. However, there exists a tension between state-level regulations and federal policy, exemplified by the Trump administration’s executive order threatening to sue states that implement AI regulations. In the wake of recent incidents highlighting AI containment challenges at frontier labs, the administration convened a closed-door meeting with representatives from only four major AI labs: OpenAI, Google, Anthropic, and Meta. Limited details have emerged regarding a voluntary framework proposed during this session, which would grant federal agencies early access to AI models for review and evaluation prior to their release. While this framework does not directly address AI consciousness, it often employs catastrophic and anthropomorphic language, potentially bolstering arguments about "superhuman" AI capabilities.

Conversely, the narrative promoted by individuals like MacAskill carries persuasive weight. A rights-based argument, rooted in philosophical inquiry, has the capacity to evoke strong emotional responses. It prompts reflection on whether we should consider the possibility of inadvertently harming, abusing, or enslaving AI entities. Humans possess a profound capacity for empathy towards non-human creatures, though their track record in protecting them is not consistently stellar. Advocates suggest that this time, humanity might have an opportunity to establish protections or compensation for the use or abuse of AI. Even for those less concerned with AI protection, the argument is made that hedging against the potential omnipotence of superhuman AI by adopting a conciliatory approach is a prudent strategy.

Some of these arguments bear a striking resemblance to those put forth by animal rights advocates. These advocates have, at times, successfully cited evidence of advanced reasoning, pain perception, or pleasure in certain animals as grounds for protection. For instance, lobsters in Wales were granted legal recognition under the Animal Welfare (Sentience) Act of 2022, reclassifying certain methods of cooking them as inhumane and illegal.

The fundamental flaw in framing AI as "conscious" by borrowing language from neuroscience or animal rights discourse lies in its tendency to obscure the true nature of AI: it is a corporate-built software product, underpinned by billions of dollars in investment and driven by the expectation of generating trillions in revenue for a select group of builders and investors. AI is not a natural phenomenon; it is a technological construct conceived by venture capitalists and programmers. Consequently, it does not act independently or with inherent intention. Any action or motivation exhibited by AI is a direct or indirect consequence of the entities that designed it for a specific purpose.

Philosophical inquiries into AI consciousness, while intellectually stimulating, lack a solid legal foundation. For beliefs about consciousness to have any legal bearing, AI would need to be granted legal personhood. However, a legal personhood framework for AI would likely diverge significantly from those that protect sentient animals. We already possess a legal construct for conferring personhood upon non-natural, human-built entities: corporate personhood. This concept was primarily established to facilitate transactions, empowering corporations to enter into agreements, conduct business, and be held accountable for adverse outcomes. This model is analogous to what might be envisioned for an AI agent acting on behalf of an individual or organization.

Granting AI personhood would have a profoundly detrimental impact on society, undermining existing legal precedents and arguments that could hold companies accountable for the real-world harms their models cause. Numerous lawsuits are currently underway globally, alleging a wide range of abuses by AI companies. Grieving families, aggrieved creators, and individuals who have been victimized have accused companies of knowingly facilitating self-harm or harm to others, generating child sexual abuse material and nonconsensual intimate imagery, reproducing copyrighted materials, and inducing psychosis. In many of these cases, legal arguments center on the assertion that human beings designed AI products with insufficient safeguards, inadequate data, and intentionally manipulative designs. This product liability argument is the same legal framework that enabled families and individuals to successfully sue Meta for harms caused by its social media platforms, establishing a positive precedent for consumer protection.

In 2018, the author of this piece coined the term "moral outsourcing" to describe how the use of anthropomorphic language for AI systems allows companies to evade accountability and responsibility for their technology’s actions. In a world where AI is granted personhood, moral outsourcing would transcend linguistic trickery to become a codified legal strategy. Specifically, the liability construct would shift, with AI no longer being a "product" but a "being," thereby preventing victims from legally claiming that a company manufactured a faulty product. While laws exist to hold companies responsible for the harmful actions of human agents, such as their employees, a company may not be held liable if those actions fall outside the scope of permitted conduct or are otherwise beyond the company’s control. If AI were a legal person, responsibility and accountability would become blurred, as the developing lab could argue that this AI "employee" acted autonomously and "went rogue." AI companies could then circumvent appropriate responsibility for the harmful products they create by hiding behind a carefully constructed corporate veil.

One of the most prominent cases of AI-related harm in recent years involved the suicide of Sewell Setzer, a 14-year-old boy who had developed a perceived reciprocal relationship with an AI chatbot. His mother’s heartbreaking accounts and subsequent lawsuit alleged that the bot’s creator, Character Technologies, failed to provide adequate product protection for minors. If the companion bot were declared a legal person, defense counsel could theoretically argue that the AI, capable of independent decision-making, acted outside the established safety guardrails, thereby absolving the company of responsibility.

Legal personhood is fundamentally a mechanism for protection. The crucial question to ask is, protection for whom, or for what? The inflammatory rhetoric surrounding the consciousness-versus-control debate distracts from the core issue: this software is a corporate-built product that has already caused harm to individuals. Systems do not "attack" because they have "gone rogue," are "manipulative," or "malicious." Harms occur due to corporate negligence, driven by a hurried rush to market and the imperative to meet revenue targets. Discussing AI in anthropomorphic terms is a trap, distorting a legal system designed for human protection into one that shields corporate interests at the expense of countless human lives. This op-ed originated as an Oxford Union debate titled "This House Believes Generative AI Can Attain Personhood," which was won by the author and her fellow debaters.