The Error-Correction Gap
Why Darwin is not the only alternative to decay
Robin Hanson has identified a serious problem. Modern civilization has become unusually good at insulating itself from failure: famine is rarer, war is less frequent among wealthy states, dysfunctional institutions can survive for decades, and rich societies can subsidize practices that would once have destroyed the populations depending on them. Cultural convergence has also reduced the number of genuinely independent social experiments. Many of these developments are achievements, but they also weaken selection pressure.
Hanson’s concern is that cultural evolution once constrained societies more harshly. Cultures that failed to reproduce, defend themselves, feed themselves, or maintain social order tended eventually to disappear. Modernity weakens those constraints while accelerating cultural change, allowing norms to spread globally before their long-run consequences are known. Falling fertility is his clearest example: a culture can become wealthy, technologically sophisticated, politically stable, and prestigious while simultaneously becoming less capable of reproducing itself.
Hanson concludes that durable values must therefore be embedded in an adaptive culture, and that Darwinian evolution is the only demonstrated force capable of resisting long-run decay. The diagnosis is compelling, but the underlying principle is broader than Darwinian evolution. It is persistence.
Values need carriers
Any value that continues to exist must be instantiated by something capable of carrying it forward. A language dies with its last speaker. A scientific tradition needs not only scientists but institutions, records, and successors willing to take it up; a moral tradition needs agents who can learn it, transmit it, and act on it.
That carrier may be a population, an institution, a culture, a state, or a cognitive lineage, and the constraint binds at whichever level the value is actually instantiated. Persistence at one level routinely coexists with replacement at another: a subculture vanishes while its host civilization continues, a state collapses while its language survives.
This is the argument developed in The Ultimate Metagame. Before a system can pursue truth, liberty, prosperity, beauty, justice, or any other goal, it must remain capable of continuing to instantiate those goals. Persistence is therefore not the highest value. It is a constraint on every value that hopes to survive.
A civilization may coherently value freedom more than reproduction. But if the institutions expressing that freedom reliably produce their own disappearance, eventually there will be nobody left to express it. Hanson is therefore right about something fundamental: a value system that systematically destroys its own conditions of continuation cannot persist indefinitely. Nothing in that proposition makes reproductive fitness our objective function.
Darwinian evolution is one mechanism
Natural selection is an extraordinarily powerful mechanism for discovering persistence-compatible structures. Its procedure is brutal:
variation → real-world testing → replication or extinction
Organisms embody different strategies. The environment tests them, and strategies that repeatedly fail disappear with the organisms carrying them. Cultures have historically faced analogous pressures. Agricultural systems that could not feed their populations collapsed, states unable to defend themselves were conquered, communities unable to reproduce shrank or vanished, and institutions badly mismatched to their environments were eventually replaced.
Hanson is right that modern civilization has weakened many of these mechanisms. But Darwinian selection is not the underlying principle. It is one historically dominant way of satisfying the deeper persistence constraint.
Intelligence introduces another.
Intelligence can move selection inside the system
An intelligent agent does not need to die every time one of its beliefs is wrong. It can revise the belief. Science does not normally settle disputes by waiting for scientists holding false theories to go extinct; it constructs rival hypotheses, subjects them to criticism and experiment, and discards those that fail. Engineering does not build a thousand bridges and identify sound designs by watching the rest collapse; it uses mathematics, prototypes, simulation, testing, safety margins, and failure analysis. Software systems use tests, monitoring, rollback, redundancy, isolation, and revision so that defective components can fail without taking the entire system with them.
The same principle appears in institutions. Markets can redirect capital without requiring whole societies to collapse. Political systems can replace governments without civil war. Prediction mechanisms can force competing models to expose their expectations before reality settles the dispute. In each case, some of the selective work that would otherwise occur through catastrophic external failure is performed internally.
The general pattern becomes:
variation of models → prediction and testing → revision
Evolution kills organisms that embody bad models. Intelligence can kill the models instead.
One can stretch the word Darwinian until every process involving variation and differential rejection counts as Darwinian. But then the claim that Darwinian evolution is the only anti-decay mechanism becomes true by definition and loses explanatory value. The useful distinction is between external selection among whole systems and internal error correction within systems. Modern civilization increasingly attempts to depend on the second.
The error-correction gap
This reframes Hanson’s problem. Historically, many failures were corrected by catastrophe:
bad agriculture → famine
bad government → defeat or revolution
bad institutions → collapse
low-fertility culture → demographic replacement
We should not romanticize these mechanisms. Their costs were immense, and avoiding them is progress. But eliminating a correction mechanism creates an engineering obligation: something else must perform the informational work the old mechanism performed. If bankruptcy is prevented, some other mechanism must identify insolvency. If political failure no longer produces rapid institutional replacement, some mechanism must expose and reverse bad policy. If cultural practices can persist for generations while undermining reproduction, societies need ways to detect the mismatch before demographic replacement does it for them.
Modern civilization has therefore created an error-correction gap. External correction has weakened faster than we have demonstrated the reliability of our internal substitutes. Modernity has not escaped the persistence constraint, since nothing can. It has postponed some consequences while increasing the importance of detecting failure by other means.
This does not mean that markets, decentralization, pluralism, or scientific criticism already contain the solution to every persistence problem. They plainly do not. Prosperous liberal societies possess all of these institutions and still exhibit severe fertility decline. Their role is more modest and more important: they create the possibility of searching among alternatives without requiring civilization-wide failure.
The civilizational challenge is therefore not to design the correct social order in advance. It is to preserve enough independent variation, feedback, and selection among institutions that workable arrangements can be discovered before failure becomes terminal. If we already knew the correct family policy, political structure, cultural norm, or institutional design, we would have much less need for error-correcting systems in the first place.
Prosperity can create locally rational extinction
Fertility makes the problem unusually clear because no individual irrationality is required. One plausible mechanism, developed in The Prosperity Paradox, is that prosperity expands optionality. It increases geographic mobility, career choice, consumption possibilities, control over time, relationship alternatives, and the capacity to live independently. Children move in the opposite direction: they impose dependence, irreversible commitments, financial costs, geographical constraints, and large opportunity costs measured in both time and foregone alternatives.
Digital modernity adds another effect. Increasing portions of social, sexual, entertainment, and status needs can be satisfied without entering the embodied institutions that historically generated courtship, marriage, community, and children. The resulting behavior can be entirely intelligible:
prosperity → more optionality → higher cost of irreversible commitments → lower fertility
No individual needs to want demographic decline, and no ideology of extinction is required. Individuals can optimize successfully against the incentive structure immediately facing them while collectively producing a civilization that does not reproduce itself. The failure can be locally rational and globally non-persistent.
This strengthens Hanson’s argument. Cultural maladaptation need not result from stupidity, decadence, or incoherent preferences. It can emerge from an equilibrium in which individually advantageous strategies collectively undermine the system that makes those strategies possible. Cultures and subcultures that avoid this equilibrium will, other things equal, constitute a larger share of the future.
But internal error correction does not require us to know in advance which alternative will work. Different religious communities, family structures, housing arrangements, labor practices, tax regimes, social norms, and local institutions can function as competing experiments. Some will fail. Others may discover ways of reconciling prosperity and autonomy with demographic continuation. The function of pluralism is not to guarantee a particular answer but to prevent one untested answer from becoming universal.
Politics has no unit test
There is an important limit to the engineering analogy. In software, an error can often be specified objectively: the system crashes, violates an invariant, or produces the wrong result. Political and cultural systems contain contested objectives. One group’s success may be another group’s failure, and disagreement over immigration, redistribution, family norms, inequality, religion, or social organization cannot be reduced to a single agreed objective function.
Error correction in politics therefore cannot mean convergence on universally accepted values. It means exposing empirical predictions to outcomes, making tradeoffs visible, preserving competing institutional arrangements, and identifying contradictions between stated goals and the conditions required to achieve them. Values may remain plural even when facts do not.
A society could, for example, knowingly choose extreme individual autonomy despite understanding that it will produce long-run demographic contraction. That is not an epistemic error merely because another society prefers demographic growth. But a society that intends its institutions to continue indefinitely while maintaining conditions that mathematically eliminate the population carrying them contains a contradiction between its goals and its model of reality.
The same applies outside demography. People can disagree legitimately about how much debt a government should carry, but they cannot make arithmetic optional. They can disagree about the acceptable costs of an energy policy, but not about whether energy must physically be supplied. Internal error correction does not settle moral disputes. It prevents factual and structural constraints from disappearing inside them.
Persistence is not morality
Imagine two civilizations. One is scientifically productive, peaceful, prosperous, liberal, and reproductively sub-replacement. The other is authoritarian, epistemically closed, poor, and reproductively prolific. If those conditions persist long enough, the second may replace the first demographically.
Evolution can explain that outcome. It cannot justify it. Cancer persists. Parasites persist. Brutal institutions can persist. Successful replication demonstrates compatibility with a selection environment; it does not establish moral worth.
Yet the converse constraint remains unavoidable. If we value a civilization, institution, practice, or way of life, its ability to continue matters, because disappearance terminates the values it instantiated.
This is where Hanson comes closest to sliding from description into normativity. “Adaptive” can mean simply capable of persisting under relevant selection pressures. Once adaptiveness begins to function as the master criterion of civilizational success, the argument has moved too far. A successful parasite is adaptive. A totalitarian fertility cult may be adaptive. Neither follows as desirable. The defensible conclusion is only this: values must remain compatible with the continued existence of something capable of carrying them.
Persistence constrains morality. It does not define it.
Correction must arrive before lock-in
Internal error correction has another limitation: feedback can arrive too late. Many civilizational failures unfold slowly, and some become difficult to reverse long before their consequences are undeniable. Demography is an obvious example. A society cannot respond to a missing generation by immediately manufacturing millions of thirty-year-olds. Capital stock, institutional competence, environmental damage, debt burdens, and technological dependencies can exhibit similar path dependence.
An error-correcting system therefore needs more than accurate retrospective measurement. It needs prediction and early warning:
prediction → early warning → reversible experimentation → correction before lock-in
This is where intelligence has an advantage that blind selection does not. Natural selection discovers failure after organisms encounter it; intelligent agents can sometimes model the consequence before paying its full price. The advantage is not merely that intelligence can correct errors, but that it can sometimes correct them before reality makes them irreversible. An error-correction system whose feedback arrives only after the point of no return has not escaped catastrophic selection. It has merely instrumented it.
That places a premium on leading indicators, competing forecasts, scenario analysis, experiments small enough to reverse, and institutions capable of acting before consensus becomes overwhelming. Waiting until everyone agrees that a system has failed is often equivalent to waiting until correction has become expensive.
Detection is not correction
Foresight is useful only if institutions can act on what they discover. A society may correctly identify a problem decades before catastrophe and still fail to change course, because the costs of correction are concentrated in the present while the benefits arrive in the future. Politicians face elections, firms face quarterly incentives, voters discount future burdens, and organized beneficiaries defend arrangements whose long-run costs are dispersed across people who are not yet politically powerful or not yet born.
This creates a second gap, between knowledge and adaptation. A civilization can possess excellent models and weak corrective incentives. Sovereign debt, unfunded obligations, infrastructure decay, and fertility decline can all become highly legible without becoming politically tractable. The signal exists; the mechanism for acting on it does not.
Internal error correction therefore requires more than epistemic competence. It requires an incentive structure in which detecting a failure can lead to changing the system that generates it. Otherwise forecasting merely turns catastrophe from an unforeseen event into a predicted one. This is the Cassandra problem of modern governance: knowing what is coming does not imply being able to prevent it, and a system capable of accurate prediction but incapable of acting on inconvenient predictions remains poorly corrigible.
Pluralism and urgency
Pluralism introduces a further tension. Distributed experimentation is valuable precisely because we do not know which institutions or policies will work. But distributed discovery can be slow, while some failures have deadlines. Demographic structure, ecological thresholds, fiscal crises, war, and technological risks can create points beyond which options narrow rapidly.
Urgency therefore creates a powerful temptation toward centralization. If a problem must be addressed quickly, coordinating action across an entire society may seem more attractive than waiting for decentralized experiments to converge. Sometimes coordination is genuinely necessary, and a system that refuses all coordinated action in the name of pluralism can fail just as surely as one that suppresses all variation in the name of efficiency.
The distinction that matters is between discovery and execution. Discovery should remain as plural and independent as the problem permits; coordination should be no broader, more coercive, or more durable than the demonstrated constraint requires. A society may need common action once a constraint is well established, but it should resist turning that need into a permanent license to suppress competing models and experiments.
Centralization can increase speed while reducing epistemic diversity. Decentralization can increase variation while reducing coordination. There is no universal optimum. The architectural requirement is to preserve independent error correction for as long as uncertainty remains, and to avoid making an emergency response more universal or permanent than the underlying problem demands.
Replace catastrophe with criticism
The civilizational challenge is not to restore the brutality of old selection regimes. It is to preserve their error-correcting function while avoiding their cost, by creating conditions in which unsuccessful ideas, policies, and organizations can fail without requiring entire societies to fail with them.
Competition, exit, reversible experimentation, scientific criticism, prediction, and failure containment are useful because they create variation while keeping failure local. They permit one jurisdiction to discover that a policy is bad without requiring every jurisdiction to discover it simultaneously. They permit firms to disappear without taking the economy with them, hypotheses to die without killing their proponents, and governments to be removed without destroying the political order.
The common principle is corrigibility. A system should expose errors quickly, permit alternatives to exist, and allow failed components to be replaced before their failures threaten the containing system.
This is one reason monoculture is dangerous. When a single institutional or ideological architecture suppresses exit and alternative experiments, errors become correlated, and a mistake can propagate through the entire system before reality has an opportunity to discriminate among competing approaches. What makes pluralism valuable is not the number of nominal institutions but the independence of their error modes: ten regulators sharing one model and one set of incentives are not meaningfully plural. Pluralism is therefore not merely a political preference. It is an error-correction architecture.
The same logic applies to concentrated political power. A system that centralizes authority while insulating policy from competition and exit is epistemically fragile because it increases the scale at which a single mistake can become mandatory. The design problem is not maximizing decentralization as an end in itself. It is preventing errors from becoming universal before they can be discovered.
Artificial intelligence can widen or close the gap
Hanson worries that artificial minds may inherit values produced by a temporarily maladaptive human culture and then stabilize them. That possibility deserves attention, but artificial intelligence also makes the distinction between selection and error correction more important.
Artificial agents may be capable of evaluating alternatives internally at scales biological organisms never could. They can potentially simulate policies before executing them, preserve competing models, maintain explicit uncertainty, run adversarial critics, compare forecasts with outcomes, and revise themselves without waiting for demographic replacement. Used well, such systems could dramatically reduce the cost and latency of discovering mistakes.
They could also produce the opposite outcome. If nominally independent governments, firms, researchers, and institutions outsource judgment to the same small set of models, apparent pluralism can conceal deep epistemic monoculture. Ten thousand decision-makers consulting the same mistaken model do not constitute ten thousand independent experiments. The independence of error modes is exactly what a shared cognitive substrate erodes.
AI may therefore increase error-correction capacity while simultaneously increasing correlated-error risk. Whether it closes or widens the gap depends partly on whether artificial cognition produces genuinely independent models, adversarial perspectives, and competing institutional experiments, or merely distributes the same cognitive architecture everywhere.
The central question is not whether advanced intelligence can escape selection. It cannot escape reality. The question is whether it can internalize enough selection that catastrophic external failure becomes increasingly unnecessary, without converting civilization into a single brittle cognitive monoculture.
A sophisticated civilization should discover that a policy is destructive before collapse proves it. An artificial agent should discover that a model is wrong before acting irreversibly on it. A political system should discover that an institution has failed before revolution becomes the only repair mechanism. Intelligence makes these things possible without guaranteeing them.
An unproven civilizational capability
There is an important asymmetry in the argument. We have many examples of intelligent systems replacing catastrophic selection in bounded domains. Engineers detect structural weaknesses before bridges collapse. Scientists discard failed hypotheses without eliminating the scientific community. Financial systems can liquidate individual firms instead of allowing every commercial mistake to propagate through the economy. Constitutional succession can replace rulers without requiring civil war.
What we do not yet have is comparable evidence that an entire prosperous civilization can reliably detect and reverse a slow, distributed persistence failure before external selection does the correcting. Demographic decline is especially revealing, because no modern affluent society has yet demonstrated a durable return from very low fertility to replacement-level reproduction through deliberate internal correction while preserving the broader conditions of modern prosperity.
That uncertainty should not be hidden. The claim that intelligence can in principle move selection inside the system is well supported. The claim that modern civilization has already built internal mechanisms adequate to its deepest persistence problems is not.
The error-correction gap is therefore not a solved engineering pattern. It is an unsolved civilizational problem. Modernity has weakened several brutal but effective forms of external selection and is betting that prediction, experimentation, pluralism, institutional revision, and deliberate coordination can eventually perform the same corrective function at lower cost.
We do not yet know that the bet will succeed.
Postscript
Hanson has identified a genuine vulnerability of modern civilization. We have become unusually good at insulating ourselves from consequences that once destroyed maladaptive systems. That insulation has prevented enormous suffering, but it has also weakened some of the mechanisms through which civilizations learned that they were failing.
The mistake would be to conclude that we therefore need harsher selection. We need better correction, and there is no guarantee that we have yet built enough of it.
Darwinian evolution discovers viable organisms through death and reproduction because it has no model of what it is doing. Intelligent systems can represent alternatives, criticize them, test them, and sometimes revise before failure becomes terminal. But prediction does not imply action, pluralism can conflict with urgency, institutions can suppress criticism, governments can socialize failure, prosperity can postpone consequences, and monocultures can eliminate the independent variation on which correction depends.
Nor does this framework tell us which family policy, political order, social norm, or AI architecture will work. If it could provide those answers in advance, much of the need for error-correcting institutions would disappear. Its claim is architectural: preserve independent variation, make consequences legible, detect failures early, align incentives so detection can produce action, permit reversible experiments, coordinate only where coordination is necessary, and allow unsuccessful arrangements to be replaced without making their failure universal.
Modernity has suppressed many of the ancient mechanisms that selected against failure. It therefore acquires a corresponding obligation to build mechanisms capable of discovering persistence-compatible arrangements from within. That is not an optimistic prediction about what civilization will do. It is a condition civilization must satisfy if it wants to retain the benefits of replacing brutal external selection with cheaper internal correction without eventually being corrected by the former.
A mature civilization should not require extinction to discover that it was wrong.