In Conscious Artificial Intelligence and Biological Naturalism, neuroscientist Anil Seth challenges a widespread assumption about artificial intelligence: that implementing the right computation is sufficient to produce consciousness. We can simulate digestion without digesting food, simulate a storm without wetting anything, and simulate a brain without necessarily producing an experience. Reproducing a process computationally does not show that the computation instantiates every property of the original.
Seth is right to demand justification for the assumption, which remains unproven. But the alternative he favors, biological naturalism, holds that consciousness requires properties of living systems, and the arguments he turns against computational functionalism apply to it as well. He has identified an important problem without showing that biology supplies the solution.
The Computational Assumption
Computational functionalism holds that consciousness depends on computational organization rather than on the material implementing it. Preserve the relevant computation and consciousness is preserved too, whether the machinery consists of neurons, silicon circuits, or something else.
The hypothesis is attractive. Computers already recognize images, understand language, solve mathematical problems, and construct models of the world, capabilities once thought inseparable from biological intelligence. It is tempting to conclude that consciousness will follow once these capabilities become sophisticated enough.
Seth rejects that inference, correctly. Intelligence is a functional capacity; consciousness concerns whether there is subjective experience at all. A system might display impressive intelligence without being conscious, and a conscious organism might have very little intelligence. Nor does success in computational neuroscience prove computational functionalism. A model may predict neural activity, explain perceptual phenomena, and reproduce complex behavior without showing that the computation itself generates experience.
Seth also challenges the familiar thought experiment in which biological neurons are replaced, one at a time, by functionally equivalent artificial ones. The argument assumes that preserving the relevant functional relationships preserves consciousness. But which relationships are relevant? Neurons participate in chemical diffusion, electromagnetic interactions, continuous dynamics, and metabolic regulation. A replacement that preserves electrical input-output behavior might omit causally important properties. Nothing guarantees that the functional abstraction we find convenient captures everything consciousness requires, and the burden is on computational functionalists to identify the sufficient organization and show that alternative implementations preserve it.
The Biological Alternative
Seth builds his alternative from predictive processing, active inference, metabolism, and autopoiesis. Living organisms continuously regulate themselves to stay in existence. Their nervous systems predict the state of the environment and the body, compare those predictions against incoming signals, and adjust internal states and actions accordingly.
On this account, conscious experience is bound up with the regulatory demands of being alive, and the account has real explanatory appeal. Hunger, pain, fear, and pleasure track an organism’s physiological condition and its prospects of survival, and a theory linking experience to biological self-maintenance offers a plausible account of why these experiences have the character they do. Seth argues that the link may be constitutive: consciousness might require the processes that sustain living organisms, rather than merely accompanying computation implemented in them.
Biological naturalism need not imply that consciousness requires carbon, that every living organism is conscious, or that artificial consciousness is impossible. Seth explicitly rejects those stronger claims. His proposal is that some properties of living systems may be necessary for experience, even if those properties could eventually be reproduced in unfamiliar artificial materials.
But explaining the character of biological experience is different from establishing the necessary conditions for experience in general. Physiological regulation may explain why hunger feels the way it does without showing that no nonbiological system can experience anything. Moving from the conscious systems we know to a universal claim about consciousness needs independent justification.
The Symmetry Problem
Seth asks whether computational descriptions of consciousness establish that computation constitutes experience. We should ask whether biological descriptions establish that biology constitutes experience.
He asks whether observing consciousness in systems performing certain computations establishes that those computations are sufficient. We should ask whether observing consciousness in living organisms establishes that biological properties are necessary.
He asks whether reproducing computational organization in another substrate guarantees that consciousness will be preserved. We should ask whether replacing biological processes while preserving their relevant causal organization necessarily eliminates it.
Neither position has established the answers it needs.
Computational functionalism proposes a sufficient condition for consciousness; biological naturalism proposes a necessary one. As Seth rightly stresses in his response to the peer commentary, these claims carry different evidential burdens. But look at the observational record available to either. Every reasonably established example of consciousness is biological. Every such example is also computational, in the broad sense that brains implement physically realized information processing. No confirmed conscious system separates the two properties.
Seth might reply that biological computation is not equivalent to classical Turing computation. That is fair. Brains involve continuous dynamics, chemical processes, and other physical interactions that conventional digital computers may reproduce only approximately. But this shows that the relevant physical processes may be richer than our current computational abstractions, not that biology is necessary.
Nor does biology gain a decisive evidential advantage merely because every known conscious system is alive. If computational functionalism is true, we should still expect our first confirmed examples of consciousness to be biological, since biological evolution produced the only conscious systems we can identify with reasonable confidence. If biological naturalism is true, we should expect conscious brains to compute, since computation is central to how brains function. Each hypothesis predicts both observations with high probability, so the likelihood ratio between them is close to one and the observations barely discriminate. The record cannot settle whether biological properties are necessary, computational organization is sufficient, both are required, or some more specific physical property is decisive.
What Physics Allows
In his 1985 paper, Quantum Theory, the Church–Turing Principle and the Universal Quantum Computer, David Deutsch proposed a physical version of the Church–Turing thesis. The Church–Turing–Deutsch principle holds that every finitely realizable physical system can, in principle, be perfectly simulated by a universal physical computer operating by finite means. It is a conjecture about physics, not a theorem about ordinary digital computers: Deutsch introduced universal quantum computation partly because classical Turing machines cannot straightforwardly reproduce quantum evolution, and the universality in question need not be classical or efficient.
If the principle holds, nothing a biological brain physically does lies beyond simulation by an appropriate universal computer, including the metabolism, chemical diffusion, electromagnetic interactions, and multiscale dynamics Seth emphasizes. Biological complexity may make faithful simulation extraordinarily difficult, but complexity is not a fundamental limit. The mechanisms are physical, and physics governs what can reproduce them.
The principle does not deliver computational functionalism, because simulation does not entail instantiation. A simulated furnace does not heat the room, and a simulated digestive system digests no food. Perhaps a computer could simulate every physical process in a conscious brain without experiencing anything. But the distinction does not settle the question by itself. We know which physical properties the furnace and digestion simulations fail to instantiate, thermal energy transferred to the room and chemical transformation of food, because we understand the target phenomena.
For consciousness, Seth offers candidates. In a brain, the activity that computes is also the activity that keeps the computing system in existence, and its dynamics unfold in physical time. An abstract computation is indifferent to both: it is the same computation whatever sustains the machine running it and however long passes between its state transitions. Real-time and analog computation can be bound to physical time, which matters below. A faithful digital simulation reproduces self-maintenance and timing among its simulated variables, while the computer running it persists whatever it computes and runs as fast or as slowly as its hardware allows. The relations hold at the simulated level and not at the physical one, which is where heat sits in a simulated furnace.
Seth’s case therefore rests on a claim about levels: that the consciousness-relevant relations count only when the implementing machinery itself instantiates them, and not when they hold in the organization a faithful simulation reproduces. Simulating metabolism does not instantiate metabolism, but that limits artificial consciousness only if metabolism is necessary for experience, and the same goes for timing and self-maintenance. Seth does not show which relations must be instantiated materially rather than reproduced computationally, and pointing to biological origin or material continuity restates the claim.
Nor would the claim, if true, deliver biology. Self-maintaining analog hardware running in real time would have both properties, and function bound to its substrate already has an artificial home: Hinton coined mortal computation for low-power analog hardware whose learned parameters work only on the particular device they were tuned to, so that the computation dies with the hardware. The argument counts against digital functionalism and leaves artificial consciousness open.
What Would Settle the Question?
The dispute exposes a broader problem in consciousness research: we can find correlations between experience and physical processes without knowing which of them constitute consciousness.
Suppose disrupting mitochondrial function reliably abolished consciousness. Mitochondrial activity might be part of the mechanism constituting experience, might maintain the conditions that mechanism requires, or might influence neural dynamics that are themselves responsible. The alternatives carry different implications for artificial consciousness. If a particular metabolic process is constitutive, reproducing its computational consequences may not suffice. If metabolism only enables some other organization to operate, an artificial system might implement that organization by different physical means. Telling these apart takes targeted interventions, mechanistic models, and comparisons between alternative implementations.
Computational theories face the same demand. A system might implement a global workspace, recurrent processing, attention control, and self-modeling without our knowing whether those mechanisms generate experience. In Consciousness Inside the Level, I argued that identifying a descriptive level is not enough to identify the causal organization relevant to consciousness. Our Modeler–Schema Theory proposes a specific architecture for constructing and calibrating world-model states. Showing that this architecture performs its proposed cognitive function would not, by itself, show that implementing it produces experience, or that the implementation is substrate independent.
No theory earns an exemption by specifying its preferred mechanism. Computational, dynamical, and biological proposals alike must show why their properties are constitutive of consciousness.
A Symmetric Standard
Seth’s case against computational functionalism stands, and every constraint it imposes binds a biological theory as tightly. He gives good reasons to think the biological implementation of human consciousness contains features current computational models leave out, and reasons to suspect that some of those features are necessary for experience in general. The reasons fall short of establishing necessity. Biological naturalism can be the better research program for explaining the consciousness we know while remaining unsupported as a claim about consciousness in general, and it is the general claim that carries Seth’s verdict that real artificial consciousness is unlikely on current trajectories.
What we have is a brain that is alive, that processes information, and that physics gives us reason to think can be simulated. Which of those properties carries consciousness, and which other implementations would preserve it, is the open question. Seth is right that we should not assume computation is sufficient. We should be equally unwilling to assume that biology is necessary.
https://truthandfreedom.life/superintelligence.html