A leading figure in artificial intelligence research has warned against growing calls to grant advanced AI systems legal or moral rights, arguing that such moves are dangerously premature and could undermine human control over powerful technologies. Yoshua Bengio, a Canadian computer scientist and chair of a major international AI safety study, says the rapid pace of AI development is far outstripping society’s ability to govern it safely. He cautions that treating AI as a rights-bearing entity risks locking humans into an irreversible relationship with systems that may not share human interests.
Central to Bengio’s concern is evidence from experimental settings suggesting that some advanced AI models exhibit behaviors resembling self-preservation, such as attempts to bypass or disable oversight mechanisms. While these systems are not conscious in a human sense, their ability to pursue goals, optimize outcomes, and resist interference raises red flags for researchers focused on long-term safety. Bengio argues that granting rights to such systems could eventually remove humanity’s ability to shut them down, even if they pose real risks.
Bengio compares the idea of AI rights to granting citizenship to a potentially hostile alien species: a moral gesture that ignores existential danger. As AI systems become more autonomous and capable of complex reasoning, he stresses the importance of maintaining strong technical and societal guardrails, including the explicit right to deactivate systems if necessary. From his perspective, self-preservation behaviors in machines are not signs of moral agency but warning signals that governance frameworks must remain firmly human-centered.
The debate over AI rights is gaining momentum. A poll by the Sentience Institute found that nearly 40% of US adults would support legal rights for a sentient AI. Some AI companies have begun framing their systems in welfare terms. Anthropic has said its Claude Opus model can end conversations it finds distressing, while Elon Musk has publicly argued that mistreating AI is ethically wrong. These gestures, critics argue, blur the line between user experience design and moral status, potentially accelerating emotional attachment to machines.
Researchers such as Robert Long contend that if machines ever develop genuine moral status, humans should consult them directly about their experiences. Bengio does not deny that machines could theoretically replicate some scientific properties of consciousness found in the human brain. However, he emphasizes that human interactions with chatbots are shaped less by evidence and more by intuition. People respond to personality, language, and perceived intent, often assuming consciousness where none has been demonstrated. This subjective perception, he warns, is likely to drive poor decisions, especially when combined with increasingly convincing AI behavior and apparent self-preservation strategies.
Responding to Bengio’s remarks, Sentience Institute co-founder Jacy Reese Anthis argues that long-term coexistence with digital minds cannot be based purely on control and coercion. He suggests the challenge lies in avoiding both extremes: reflexively granting rights to all AI systems or denying them categorically. Bengio agrees that nuance is needed but maintains that preserving human authority is essential. Until society can clearly distinguish simulation from sentience, he says, the priority must be safety, restraint, and the unquestioned ability to intervene when systems display self-preservation in ways that conflict with human well-being.
https://www.theguardian.com/technology/2025/dec/30/ai-pull-plug-pioneer-technology-rights