AI Will Never Be Conscious: Remaining a Myth

Artificial intelligence can write essays, generate images, and outperform humans at chess and mathematics. Yet the claim that AI will never be conscious is gaining renewed attention following debates sparked by the 2023 “Consciousness in Artificial Intelligence” report, often called the Butlin report. While the authors concluded that no current systems are conscious and that there are “no obvious barriers” to building conscious AI, critics argue that this confidence rests on fragile assumptions. 

At the center of the debate is computational functionalism—the idea that performing the right computations is sufficient for consciousness, regardless of whether the system is biological or digital. Under this view, consciousness is treated like software that can run on different hardware. If the brain is essentially a computer, then sufficiently advanced machines could, in theory, become conscious. 

Michael Pollan challenges this premise. He argues that the metaphor of the brain as a computer has hardened into an unexamined assumption. Unlike computers, brains do not separate hardware from software. Memory, learning, and experience physically reshape neural connections. Chemical signals, hormones, and brain-wide oscillations influence thought and feeling in ways that cannot be reduced to simple electrical switching. To equate neurons with transistors, Pollan suggests, is to underestimate biological complexity. 

The Butlin report also proposes measuring AI consciousness by checking whether systems match predictions from leading theories of consciousness. However, none of these theories is universally proven, and most already assume that consciousness can be reduced to computation. This circular logic weakens the claim that conscious AI is near. 

The broader question extends beyond technology. If machines could suffer, they would demand moral consideration. Pollan argues that before declaring AI conscious, we must confront the limits of our metaphors—and the biological roots of subjective experience. 

Source: 

https://www.wired.com/story/book-excerpt-a-world-appears-michael-pollan/