Imagine a committee of ten people evaluating a proposed policy. Nine privately believe the policy is a mistake. The tenth supports it enthusiastically and makes clear that opposition would be regarded as irresponsible. One by one the others express their support, since nobody wants to be the first to disagree, and the meeting ends in what appears to be a unanimous decision.
Then one member raises an objection. Another member, who privately shares it, attacks the dissenter for undermining the committee’s work, forcefully enough to convince everyone else that the attacker must believe in the policy.
The committee has manufactured evidence of agreement from disagreement, and it has given the people who recognize the error a reason to stop anyone from correcting it. Each of the nine knows the policy is a mistake. None of them knows that the other eight know. The group holds the information needed to reject the policy and lacks common knowledge of it.
The People Who Punish the Truth
In 2009, sociologists Robb Willer, Ko Kuwabara, and Michael Macy published The False Enforcement of Unpopular Norms in the American Journal of Sociology. That people publicly endorse judgments they privately reject was already familiar. Their question was why someone would go further and punish another person for voicing the very doubts they share.
Their answer was reputation management. Someone who conforms under pressure may worry that the conformity looks insincere, and publicly criticizing a dissenter displays commitment to the norm. The authors called this false enforcement and tested it in three laboratory studies.
In the first, participants rated wines after simulated group members had established a misleading consensus. In the second, they rated a deliberately unintelligible academic text that other supposed participants had praised. In both, participants who went along with the group rated a lone dissenter below the conformists when their ratings of other participants were public. When those ratings were private, the conformers in the wine study favored the dissenter. The third study examined the audience: observers who saw someone endorse the text and also mark down a dissenter judged that person’s private opinion of the text to be more favorable than observers who saw the endorsement alone.
Each sample was a few dozen undergraduates. Private belief was never measured directly, only inferred from the difference between public and private conditions, and the authors allow that some conformers may have been persuaded. The experiments show that the behavior can be produced on demand, and they leave open how often institutions produce it.
Counting the Same Evidence Twice
Ten independent researchers who reach the same conclusion provide strong evidence, because each had the chance to find what the others missed. Ten researchers who copied one mistaken source provide a single investigation reported ten times. The evidential value of agreement depends on how probable that agreement would be under rival explanations, and the rivals include shared sources, shared incentives, and shared errors.
Conformity is a shared incentive. When people agree because they believe everyone else agrees, the tenth endorsement was largely caused by the first nine and adds almost nothing about the underlying question. False enforcement adds a second layer. A person who has conformed attacks a dissenter, and an observer takes the attack as further evidence of sincere belief. The same social pressure produced both acts, so the observer has counted one cause twice: first as agreement, then as punishment of disagreement.
The observer cannot simply apply a discount. Enforcement is evidence of conviction to the extent that sincere believers are more likely to enforce than private doubters are. How alike the two groups behave depends on what enforcement costs, whether it is required, and what silence would cost the enforcer, and public behavior conceals all three.
Later work located where the illusion holds. Minjae Kim and Ezra Zuckerman Sivan found in 2017 that enforcers who volunteer an attack, in settings where hidden deviants have reason to fear accusation, attract suspicion of being hidden deviants themselves. Enforcement read as sincere when the enforcer’s role required a judgment. Mandated judgment is how institutions review things: referee reports, performance ratings, committee votes. A mandate also removes silence as an option, since the reviewer who declines to condemn has supplied evidence against his own loyalty. The settings built to scrutinize claims are the settings where an insincere attack is hardest to tell from a sincere one.
The Knowledge Nobody Can Use
A naive account of collective intelligence expects more minds to mean more information and a better chance of catching errors. That expectation fails when what people say is priced by reputation, because public deliberation then misreports the knowledge distributed among the members. The nine hold the relevant objections and would reach the same verdict independently, yet the institution behaves as though the objections do not exist.
The nine could be wrong. Privately unpopular policies are sometimes correct, and the experiments demonstrate distortion in what people report, which is a different thing from the suppression of true conclusions. The institutional loss does not depend on who is right. A record that misstates what members believe withholds the criticism needed to test the policy, and criticism that proves mistaken is still how a sound policy gets shown to be sound.
The process can also invert the relationship between knowledge and influence. Someone who privately recognizes an error may become one of its most aggressive defenders, because an earlier public endorsement is now a commitment and admitting the error would expose the insincerity. While dissent stays concealed and standing depends on conspicuous conformity, each new doubter who remains publicly loyal is another person with sincerity to prove. Under those conditions an institution can become more resistant to correction as more of its members privately conclude that something is wrong.
Doubt also pushes the other way. Timur Kuran’s Private Truths, Public Lies describes apparent agreement collapsing quickly once a credible challenge shows people how many others share their view. Which force wins depends first on whether private doubt becomes common knowledge: a doubter who believes himself alone has every reason to keep enforcing, and doubters who can see one another have the material for a collapse. Visibility is necessary without being sufficient, because where sanctions are severe enough, people go on enforcing a rule that everyone knows most of them oppose.
The Error-Correction Failure
In The Error-Correction Gap, I argued that intelligence can in principle replace catastrophic external selection with internal error correction, and that modern civilization has weakened the external kind faster than it has built reliable internal substitutes. False enforcement is one mechanism by which the substitutes fail.
The familiar failure is an institution refusing to correct an acknowledged mistake. False enforcement fails earlier, by keeping the mistake from becoming publicly recognizable. Formal procedures for criticism and review run on what individuals are willing to reveal. If revealing a disagreement carries a reputational penalty, members conceal it, and if enforcing conformity carries a reward, they suppress it in others. The inputs are corrupted before the formal evaluation begins, which is why review can be impressive on paper and ineffective in practice. A committee of formally independent members can behave as a single conformist body, so the number of participants says little about the independence of their judgments.
The institution can then mistake the resulting silence for success. Complaints decline, reports grow more favorable, and administrators read the trend as improving performance when it reflects the rising cost of telling the truth. This is epistemic Goodharting. Reported consensus is a proxy for independent agreement among the people consulted, and once the proxy is read as a measure of institutional health, the incentive is to produce the report whatever the state of the agreement. At the limit, the institution cannot distinguish success from successful suppression of the evidence of failure.
The Architecture of Dissent
The laboratory design suggests a diagnostic. The experimenters detected false enforcement by comparing what people said when they were identified with what they said when they were not, and an institution can run the same comparison on itself. The test is asymmetric. A substantial gap between anonymous and public assessments is evidence that the conditions of expression are shaping what gets reported. A match is weaker reassurance, because anonymity does nothing about beliefs sincerely absorbed from the group or errors that everyone shares.
Other signs can be read from the record. Were judgments written down before discussion or only after it? Did anyone who dissented pay for it? Will the people endorsing a position stake a forecast on it? None of the answers depends on how confident the endorsers sound.
The remedies follow from the diagnosis. Publishing an anonymous tally converts private doubt into common knowledge. Independent assessments made before group discussion preserve the information that conformity would erase. Explicit forecasts make competing claims answer to consequences instead of to headcount, and they supply what no survey can: a match between anonymous and public views shows at most that the agreement is sincere, and only a test against outcomes shows that it is right.
Incentives need the same attention. An institution that punishes the member who reveals a mistake will lose access to inconvenient information, and a declared commitment to transparency does not change that. Protecting internal critics, allowing dissenting reports, and keeping evaluators from depending on the approval of the people they evaluate do change it. So does decentralization, since competition and exit let people test alternatives without first persuading the prevailing consensus to authorize the experiment. None of this guarantees truth. The aim is to keep any single social incentive from governing every channel through which an institution learns.
Postscript
Four years before the experiments, Damon Centola, Willer, and Macy modeled the dynamics in The Emperor’s Dilemma. In their simulations an unpopular norm could not take over a population in which everyone observed everyone, unless true believers already made up about a third of it. A handful of true believers clustered in one neighborhood could start a cascade to near-universal enforcement that hundreds scattered at random could not. A modest share of random long-range ties stopped the cascades, because those ties gave doubters a more representative sample of what other people thought.
A model is only a model, but this one says where to look. False consensus is a local product, and a tie between institutions can carry it or kill it depending on what the tie connects. Universities, newsrooms, and professional associations that hire from, read, and answer to one another replicate a judgment, so an apparent consensus passing among them gains repetitions without gaining independent judgments. Ten such institutions can amount to one committee. Ties to populations with different incentives do the opposite work, since they are how a doubter learns that the neighborhood is unrepresentative.
Where false consensus rests on ignorance of what others think, it is fragile, because a representative sample of private opinion is cheap. Where it rests on sanctions, the sample exposes it without ending it, and the incentives have to change as well. For the outside observer the argument reduces to three questions about any consensus. Are the judgments independent, or do they share sources and neighborhoods? What does dissent cost the people expressing them? What evidence stands behind the agreement apart from the agreement itself? Confidence, unanimity, and the vigor with which critics are attacked answer none of the three.
The danger is a civilization whose members know more than its institutions can admit. Such a civilization keeps producing studies, conducting reviews, and announcing overwhelming agreement while its public conclusions drift away from the judgments of the people producing them. Correcting errors requires that someone be able to report that the emperor is naked without everyone else proving their loyalty by punishing the observation. Otherwise the emperor remains fully clothed in every official report, right up until reality delivers its own assessment.


