Fallibilist Bayesianism
Bayes within models, criticism between them
Bayesian inference is one of the finest tools ever devised for reasoning under uncertainty. Given a hypothesis space, prior distribution, representation of evidence and likelihood model, it tells us how the evidence should change the relative weights assigned to the hypotheses. Within that structure, the update rule is exact.
Scientific reasoning requires more.
The hypothesis space may omit the truth. The evidence may have been represented at the wrong level. The likelihoods may encode a false causal picture. A model can be internally coherent, update correctly and remain systematically wrong about the world.
Bayesian inference moves probability mass among represented possibilities. Inquiry also creates possibilities, changes representations, diagnoses malformed questions and replaces the models within which probabilities were assigned.
This suggests a division of epistemic labour:
Bayes within models, criticism between them.
Call the resulting stance fallibilist Bayesianism. It accepts Bayesian inference wherever a probabilistic model has earned authority. It places that inference within a larger cycle of conjecture, formalization, testing, criticism and reconstruction. Here “model” includes any structured representation that determines which possibilities, observations and inferences are available to an agent, and “between models” should be read broadly: criticism tests the frame, including its variables, boundaries, evidence model and admissible alternatives, whether or not a rival frame yet exists. Every probability remains conditional on a frame, and every frame remains open to revision.
The Closed Bayesian Picture
The standard Bayesian picture begins with a set of hypotheses. An agent assigns prior probabilities to them, receives evidence, updates by conditionalization and chooses actions by expected utility.
This architecture captures something important. Beliefs should respond systematically to evidence. Confidence should be distributed consistently. Decisions should account for both uncertainty and consequence.
The difficulty lies in treating this architecture as a complete account of rationality.
A Bayesian update presupposes that the relevant possibilities have already been represented. The hypotheses must exist as elements of the model. The evidence must arrive in a form that the model can process. The boundaries of the problem must already have been drawn.
Actual inquiry repeatedly changes all of these.
A physician may discover that a familiar syndrome contains several causally distinct diseases. A physicist may replace the variables used to describe a system. A software engineer may spend hours comparing suspected failure modes before discovering that the system violates an architectural assumption no one had thought to list as a hypothesis.
In each case, progress changes the space over which inference occurs.
A sufficiently permissive Bayesian can reply that every possible model could have been included in a larger prior from the beginning. That answer secures formal universality by removing operational content. No finite agent possesses an explicit hypothesis space containing every possible ontology, explanation and future conceptual innovation. Even an abstract ideal agent would require some language in which those possibilities are represented, and the adequacy of that language would become the same problem one level higher.
“Included in principle” does no work when the hypothesis has yet to be conceived.
Hypotheses Are Constructed
Theories do not present themselves to an investigator as a complete menu. They are invented.
The invention may involve analogy, recombination, causal imagination or attention to an anomaly that existing theories classify as irrelevant. The resulting theory can later enter a Bayesian comparison. Its creation did not arise from redistributing probability among the theories already available.
This becomes clearest when a new theory reorganizes the evidence rather than merely explaining an additional observation.
A new causal model may reveal that several apparently unrelated phenomena share a mechanism. It may divide one category into several, or introduce a variable that changes which observations count as comparable. Evidence previously treated as noise can become central. Evidence previously considered decisive can turn out to be confounded.
These changes alter the likelihoods themselves. They may also alter the sample space, the observation model and the identity conditions of the events under discussion.
Bayesian inference can compare a geocentric model with a heliocentric one once both have been specified. The conceptual work required to formulate the heliocentric model, identify the relevant variables or redesign observations around its predictions lies outside the comparison.
The same structure appears in ordinary problem-solving. An engineer debugging a distributed system might begin with hypotheses involving network delay, lock contention, cache invalidation and database overload. Logs and tests shift confidence among them. Eventually she notices that two services disagree about the ownership semantics of a record. The defect was absent from the original hypothesis space because the architecture made that disagreement appear impossible.
The decisive step was representational. The engineer revised her understanding of the system before she updated within it.
Expanding the Model Class
Modern Bayesian methods relax some forms of closure.
Bayesian model comparison can evaluate different statistical or causal architectures. Hierarchical models place distributions over parameters at multiple levels. Bayesian nonparametric methods allow model complexity to grow rather than fixing the number of components in advance.
These are genuine advances. They let an agent reason over much larger and more flexible spaces than elementary textbook examples suggest. The space itself, though, still has to be constructed.
A Dirichlet process may allow an unbounded number of clusters, but clustering remains the chosen abstraction. The model still specifies what counts as an observation and which kinds of variation are admissible. It can infer another cluster. It cannot discover, from its own formal resources alone, that the observations should instead be represented as trajectories, causal processes or artifacts of selection.
An infinite model class remains a model class. Extensibility within a language does not amount to freedom from language.
These methods strengthen Bayesian inference without turning it into a complete account of conceptual invention.
Evidence Is Constructed Too
Evidence sounds like the part supplied directly by the world. In practice, evidence has structure because agents give it structure.
An instrument produces voltages, pixel values, timestamps or detector events. A measurement procedure transforms these into quantities. A statistical pipeline filters, aggregates and corrects them. A theory identifies which resulting patterns bear on which hypotheses.
Even direct perception depends on categorization. A witness sees a person leave a building. Whether this counts as evidence of guilt depends on identity, timing, access, motive and the causal story connecting departure to the alleged act. The observation alone carries no fixed likelihood ratio.
The phrase “update on the evidence” therefore conceals several prior decisions: what the observation was, which features of it matter, how measurement error should be represented, what causal connection links the observation to the hypotheses, and which alternative explanations deserve inclusion.
These are fallible judgments.
An anomalous result may reveal a new phenomenon. It may also reflect instrument failure, data contamination, selection effects or an incorrect background model. Assigning probabilities to these possibilities can help once they have been articulated. The hard anomaly often matters because none of the articulated possibilities accounts for it comfortably.
The evidence then criticizes the frame rather than simply selecting an element within it.
Evidence is theory-laden without being theory-controlled. A frame determines how observations are represented and what consequences are expected, but it does not determine which observations occur. Reality pushes back through failed predictions, unsuccessful interventions and patterns that survive changes of instrument or observer.
Construction does not imply fabrication.
Independent instruments, replication, cross-domain agreement and novel prediction matter because they reduce the freedom of any single frame to explain away failure. A model can reinterpret one awkward result. It becomes progressively harder to absorb a stable pattern reproduced by different methods under different assumptions.
The world does not hand us evidence in a finished semantic form. It still constrains which constructions survive.
Internal and External Error
Probabilistic reasoning can fail in two different directions.
Internal error occurs when reasoning violates the rules of the adopted model. The arithmetic may be wrong. Conditional dependencies may be mishandled. Evidence may be counted twice. These failures are formal and often diagnosable within the model.
External error occurs when the model itself misrepresents the problem. The event-space may exclude important outcomes. The variables may combine causally different phenomena. The reference class may be selected for rhetorical convenience.
Formal correctness offers no protection against external error.
Suppose a diagnostic system considers three diseases and calculates their posterior probabilities flawlessly. If the patient has a fourth disease absent from the model, normalization forces all probability mass onto false alternatives. The system becomes more confident as it processes additional evidence, provided that evidence is interpreted through the same closed space.
The problem is misplaced certainty generated by exhaustive assumptions that reality never accepted. An explicit “other” category can reserve probability mass for model failure, but it cannot supply the omitted disease, mechanism or treatment.
This is why coherence cannot serve as the sole standard of rationality. A coherent model can be insulated from the world by its own representational limits. Rational inquiry also needs mechanisms for detecting when the available possibilities and causal assumptions have become inadequate.
Criticism Changes the Frame
Criticism performs epistemic operations that conditionalization does not.
It asks whether the variables correspond to useful distinctions. It searches for omitted causes, equivocations and hidden selection effects. It examines which assumptions generate the conclusion and whether those assumptions survive independent scrutiny. It tests the model under interventions, boundary changes and cases outside the data from which it was built.
Criticism can also reject the question.
“Which personality type causes political extremism?” may presuppose stable personality categories and a unitary phenomenon called extremism. “What is the probability that AI causes doom?” may combine extinction, political capture, value drift and loss of human agency into one event whose boundaries change across speakers.
A numerical answer can create the appearance that the conceptual work has already been completed. Criticism reopens it.
Good criticism remains constrained by the world. It cannot consist merely of imagining that the model might be wrong. A model earns criticism when it fails predictions, requires repeated ad hoc repair, collapses under plausible boundary changes, or leaves structured residuals.
Criticism becomes productive when it points toward reconstruction. It identifies which distinction may be missing, which assumption may be false or which causal structure could explain the failure.
Bayesian updating is conservative in a precise sense: it preserves the hypothesis space while changing weights within it. Criticism permits structural change.
Causality Can Be Formalized
Some of this critical work can be represented mathematically.
Structural causal models make causal assumptions explicit. They distinguish passive observation from intervention, support counterfactual reasoning and reveal when ordinary conditional probabilities cannot answer a causal question. Causal discovery methods can search over candidate graphs and use statistical independencies or interventions to eliminate possibilities.
This narrows the distance between formal inference and model criticism.
It does not close it.
Every structural causal model begins with variables, admissible graph structures and assumptions about measurement. Causal discovery often depends on conditions such as acyclicity, causal sufficiency and faithfulness. These assumptions may be justified, approximate or false.
A causal algorithm can search within a specified family of structures. Nothing in the algorithm guarantees that the selected variables capture the relevant ontology, or that a latent process has not been omitted.
Formal causal inference gives criticism more powerful instruments, and the instruments themselves remain open to criticism.
Criticism Is Not Exclusively Human
The distinction between inference and criticism should not be confused with a distinction between machines and people.
Automated systems can perform posterior predictive checks, search for counterexamples, compare causal graphs, propose hypotheses and revise models. Theorem provers expose inconsistent assumptions. Language models can generate candidate explanations and adversarial tests.
A system participates in criticism when it detects a representation’s failure relative to a standard and uses that failure to revise the representation or the space of alternatives. Mere adaptation is not enough.
The distinction is functional. Inference changes values within a representation. Criticism evaluates the adequacy of the representation. Reconstruction changes the representation.
Human judgment currently performs much of this work because humans supply broad background knowledge, causal imagination and cross-domain transfer. Nothing in the argument grants those capacities a biological monopoly. A sufficiently capable artificial agent could practice fallibilist Bayesianism more consistently than most humans do.
Criticism is also frequently distributed across agents. Scientists, engineers and institutions divide the work of proposing models, testing them and attacking assumptions. Rival perspectives matter because agents become adapted to the frames they helped construct. A community can expose errors that no isolated reasoner would detect.
The same social machinery can stabilize shared mistakes. Conformity, prestige, publication incentives and institutional capture can protect a frame after its empirical authority has weakened. Fallibilist epistemology therefore applies to systems of inquiry as well as individual minds.
Explanation Cannot Be Reduced to Weight
Scientific theories are explanations first and entries in a probability distribution second.
An explanation identifies mechanisms, constraints and dependencies. It tells us how a system would behave under conditions that have not yet been observed. It distinguishes causal structure from accidental regularity.
Posterior probability can contribute to evaluating explanations. It cannot replace explanatory judgment, because the posterior inherits the hypotheses, priors, likelihoods and evidence representation supplied to it.
A model may dominate its competitors because every competitor is worse. That establishes comparative success within the candidate set. Whether the winning model captures the governing mechanism is a further question.
This distinction is routine in engineering. Several proposed explanations may fit an observed failure, and one may fit better than the others. The engineer still asks whether it identifies a mechanism capable of producing the failure, whether disabling that mechanism removes the problem and whether the explanation generalizes beyond the original incident.
Fit without mechanism is fragile. Mechanism without empirical contact is speculation. Explanatory success arises when causal structure, prediction and criticism reinforce one another.
A mature epistemology therefore needs standards that probabilities alone do not supply: causal grip, predictive reach, resistance to arbitrary repair, robustness across representations, compatibility with adjacent knowledge, and the capacity to generate discriminating tests. These standards remain fallible and sometimes conflict. No mechanical ranking resolves every case. Judgment enters because the world admits more structure than any single metric can preserve.
Model Checking Opens the Loop
A closed Bayesian loop moves from prior to evidence to posterior and then awaits more evidence.
Scientific practice adds another operation: it asks whether the model can reproduce the relevant patterns in the data at all.
A posterior distribution may be sharply concentrated while the model systematically misses observed patterns. Residuals may retain temporal structure. Predictions may be miscalibrated across subgroups. A model may fit historical observations while failing interventions or novel cases.
These failures call for more than another update. They may require new variables, altered dependencies, revised measurement procedures or an entirely different causal account.
The operative cycle is therefore:
construct, infer, predict, criticize, reconstruct.
Bayesian inference governs part of this cycle. It helps estimate parameters, compare represented hypotheses and choose actions. Model checking determines whether those operations remain attached to the world. Criticism diagnoses failures. Reconstruction creates the next frame.
No final model escapes this cycle. A model can become extremely reliable within a domain, supported by stable measurement, repeated testing and integration with broader theory. Its authority remains conditional on the practices that sustain it.
Fallibility does not imply equal uncertainty about everything. Some models have survived vast ranges of criticism and prediction. Others are preliminary sketches held together by verbal plausibility. Fallibilism concerns revisability, not uniform doubt.
Repair or Replacement
No universal threshold determines when a model should be repaired and when it should be replaced.
The decision remains comparative.
A local repair is justified when it preserves the model’s established mechanisms, explains the anomaly with independently motivated structure and continues to generate successful predictions. A repair becomes suspect when it adds unconstrained flexibility, invokes a new exception for every failure or protects the model by making it harder to test.
A replacement frame earns preference when it explains the anomaly while preserving the predecessor’s successes, reduces arbitrary assumptions and generates new discriminating predictions.
This is neither an algorithm nor an invitation to relativism.
Scientific judgment often compares imperfect alternatives under incomplete information. The absence of a universal decision procedure does not make every judgment equally defensible. Some reconstructions explain more, predict more, survive more and require less protection from failure.
A new frame gains authority by outperforming the old one under shared constraints imposed by the world.
The Cost of Reconstruction
Criticism is not free.
Reopening a model consumes time, computation, attention and access to alternatives. An agent that questioned every variable before every decision would never act. Rational inquiry therefore depends on provisional closure: some assumptions are held fixed because revisiting them would cost more than the errors they are likely to produce.
This gives fallibilism an operating constraint.
A model may remain worth using after its limitations are known. Newtonian mechanics is sufficient for bridge design because relativistic corrections are negligible at that scale. A diagnostic heuristic may outperform a richer model when decisions must be made in seconds. A software team may tolerate an architectural defect until its operational cost exceeds the cost and risk of redesign.
The choice to reconstruct is governed by consequence as well as truth. Agents should reopen the frame when its failures become large, systematic, decision-relevant or increasingly expensive to contain.
The epistemic question is rarely whether the model is incomplete. Every useful model omits structure. The practical question is whether its inadequacy has become more costly than replacing it.
Inference inside a fixed representation can itself be computationally expensive. Reconstruction still presents a different problem: the target representation is not yet known, so the search cost is open-ended. Bounded agents must decide where to spend criticism.
Precision Must Be Earned
Bayesian culture often treats numerical credence as the mature form of belief. A person who says “I do not know” appears less rigorous than someone who says “37 percent.”
Sometimes the number improves thought. It forces comparison, reveals hidden assumptions and prevents vague language from moving opportunistically between confidence levels. Forecasting disciplines can produce genuine calibration where outcomes recur and scoring rules provide feedback.
Other problems lack the structure required for meaningful precision.
An agent may have several plausible causal stories, no stable reference class and no defensible likelihood model. Assigning a number can still serve as a private decision aid. The number then records a policy choice under uncertainty rather than a discovered property of the proposition.
There is no virtue in pretending these functions are equivalent.
Refusing precise quantification can express epistemic discipline. The refusal should carry information: which part of the model is missing, which alternatives remain unarticulated, or which event boundary remains unstable.
“I cannot assign a meaningful probability because the candidate pathways are causally heterogeneous and the dominant variables remain unidentified” is a substantive judgment. “Anything could happen” is intellectual surrender.
Fallibilist Bayesianism permits numbers where they help and withholds them where they conceal the absence of a model.
Probability Pluralism
The word “probability” is used for several related mathematical and epistemic roles.
Credence represents an agent’s uncertainty under a description. Frequency describes distributions across repeated outcomes. Statistical mechanics uses measures over inaccessible microstructure. Quantum theory contains a physically grounded measure whose interpretation remains disputed.
Shared mathematics allows these quantities to interact. Shared notation does not establish a single metaphysical substance.
A Bayesian epistemology often begins with credence because it concerns rational belief. Fallibilist Bayesianism treats credence as one instrument within an agent’s representational system. Credences can be well calibrated, poorly grounded, deliberately imprecise or undefined because the event has not been adequately constructed.
Objective structure can constrain them. Stable frequencies, physical symmetries, causal models and well-tested theories give some assignments far more authority than personal confidence alone. An agent who assigns 0.9 to heads on an ordinary fair coin is not expressing an equally valid perspective. The world supplies structure that defeats the assignment. Rational credence should track whichever objective structures are relevant to the case: stable frequencies, known symmetries, physical measures, causal mechanisms.
Probability remains conditional without becoming arbitrary.
A Fallibilist Bayesian Agent
A fallibilist Bayesian agent asks more of itself than coherence.
Before calculating, it identifies the frame. Which events have been distinguished, and for what purpose? Which possibilities have been excluded? What converts observation into evidence? Which causal assumptions determine the likelihoods?
While calculating, it respects the formal structure. Are the probabilities coherent? Have dependencies been represented correctly? Are priors and likelihoods exposed rather than smuggled into the conclusion? Does the numerical precision exceed the precision of the inputs?
After calculating, it criticizes the output. Does the model predict observations it was not built to fit? Which residual patterns remain? Does the conclusion survive plausible alternative representations? Which omitted hypothesis could reorganize the evidence?
The process has no view from nowhere. Every criticism uses some background knowledge, vocabulary and purpose. Those resources are themselves open to criticism. This creates neither paralysis nor infinite regress. Agents work from their current position, expose the assumptions most relevant to the decision and revise when failures become visible.
Inquiry proceeds through local reconstruction rather than total foundation.
Between Popper and Bayes
Popperian criticism captures something Bayesian confirmation theories often neglect: knowledge advances through conjectures exposed to possible failure. A theory earns standing by surviving serious tests and by solving problems that defeated its predecessors.
Bayesianism captures something simplified versions of falsificationism neglect: evidence usually shifts comparative confidence without deductively refuting a theory. Measurements are noisy, auxiliary assumptions can fail, and rival explanations often predict overlapping observations. Degrees of support matter.
Each approach supplies part of the machinery.
Simplified falsification cannot explain ordinary learning under noisy evidence. Pure conditionalization cannot generate or revise the representational framework within which the evidence acquires meaning.
Fallibilist Bayesianism joins them through scale. Bayesian inference handles uncertainty among articulated possibilities. Critical rationalism governs the exposure and reconstruction of the articulation itself.
The result preserves the mathematics and rejects its imperial expansion into a total philosophy of mind.
Adequacy Is Purpose-Relative
A model need not represent every real feature of a system to be adequate.
Newtonian mechanics omits relativistic structure, yet remains the right model for most bridge design. A subway map distorts geography while preserving the relations needed for navigation. A medical triage rule may ignore causal detail because its purpose is to allocate attention quickly rather than explain disease.
Model adequacy is indexed to purpose, scale, consequence and available resources.
This does not make truth optional. A representation succeeds because what it preserves is real enough to support the task. Purpose determines which structures must be preserved and which can be discarded.
The same model can be adequate for prediction and inadequate for intervention, adequate for local control and inadequate for extrapolation. A black-box predictor may outperform a causal model on next-week forecasting while offering little guidance about what would happen under policy intervention. A detailed model may explain more while imposing costs that make it unusable in real time.
Criticism must therefore ask two questions: whether the model tracks the world, and whether it preserves the parts of the world that matter for the present use.
External error is purpose-sensitive. A model does not fail merely because it omits structure. It fails when the omitted structure matters to the task for which the model is trusted. And the purpose is itself part of the frame: a purpose can be misspecified or too narrow, and it may also require criticism.
Conditionalism as the Wider Frame
Every probability is conditional on more than evidence.
It depends on an event-space, a reference class, a model boundary, a position and a purpose, as argued in Probability After Probabilism. These conditions determine which world is being compressed into the number.
Conditionalism makes those dependencies explicit. It asks what has been held fixed, which distinctions have been selected and what practical role the resulting representation serves.
Fallibilist Bayesianism is the epistemic discipline appropriate to that picture.
Bayesian reasoning tells an agent how beliefs should change while the conditions remain fixed. Criticism asks whether they should remain fixed. Model construction supplies alternatives. Testing exposes contact and failure. Judgment determines when a compression remains adequate for its purpose.
The agent never reaches an unconditional probability distribution over reality as such. It develops increasingly effective ways of representing particular structures from particular positions.
Some of those representations become extraordinarily stable. They survive criticism, support intervention, unify observations and permit precise prediction. Their success deserves confidence. Their continued openness to correction remains part of that success.
Postscript
Rationality cannot be identified with possessing a probability for every proposition.
A rational agent must know how to formulate propositions worth evaluating. It must distinguish uncertainty inside a model from uncertainty about the model. It must recognize when evidence challenges a hypothesis and when it challenges the language in which the hypotheses were expressed. It must update when updating is warranted and reconstruct when the frame has failed.
Bayesian inference provides discipline within a world-under-a-description; criticism tests the description. Fallibilist Bayesianism keeps both operations in view. It treats probability as a powerful local calculus, explanation as structured contact with causes, and every model as an achievement that may eventually need replacement.
Bayes within models.
Criticism between them.