
We need more goat sacrifices
Audio playback is not supported on your browser. Please upgrade.
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Adam Mastroianni
Nov 10, 2026
43
5
4
Share
Article voiceover
0:00
-24:56
Audio playback is not supported on your browser. Please upgrade.
photo cred: my dad
Just a quick thing before this week’s post. If you’ve preordered THE STRANGEST IDEA IN THE WORLD, I’d like to send you a little something extra in the mail. Submit your preorder receipt and your mailing address here. This is only available to folks with US or UK addresses—apologies to those who live elsewhere! In the US, everyone with a valid entry will get one; in the UK they’ll pick winners randomly. The link will be open until December 11, and we’ll start mailing things out after that. I won’t spoil the surprise, but I will say that a) it’s very silly, and b) I can’t wait to get mine.
(For the US: US Residents, 18+. Ends December 11, 2026. See terms at https://sites.prh.com/strangestidea-preorder)
Before anybody enacts a major social policy or launches any philanthropic programs, they should sacrifice a goat. There ought to be an entire liturgy of grotesqueries, in fact—pricked fingers, names written in blood, robes, candles, chanting, obsidian blades, still-beating hearts, the whole shebang.
The point of this ritual is not to signal our obeisance to the gods (although if any of them want to stop by and help us out, that would be cool), but to remind ourselves that we know nothing. This is procedurally-induced humility, which is desperately necessary, because an alarming number of our best-intentioned interventions do not work.
In 2009, Congress spent $3 billion on “School Improvement Grants” for the worst-performing schools in the country. As far as we can tell, those grants did nothing.
Universities that hired a Chief Diversity Officer between 2001 and 2019 did not end up with more diverse students, faculties, or administrators.
The American government spent $280 million to help Afghan women get jobs, and now they can’t really say whether any women got any jobs.1
Remember 20 years ago, when giving laptops to kids in developing countries seemed like the most important thing to do? Well, a 10-year RCT of 531 rural schools in Peru found zero effects of “One Laptop Per Child” on academic performance.2
Researchers developed four different messages that were supposed to convince parents to vaccinate their kids. None of the messages worked, and one of them accidentally made parents more likely to believe that vaccines cause autism.
Most of those are left-coded failures, so here are some right-coded failures:
When Arkansas added a work requirement to Medicaid, the number of Arkansans with health insurance went down without any increase in employment.
In 2002, the Bush administration launched the Building Strong Families Program, which provided counseling to low-income, unmarried parents with the goal of strengthening their relationships, keeping them both in their child’s life, and ultimately getting them married. Three years later, an independent evaluation found that the program did none of those things.
And here is the effect of four different abstinence-only sex ed programs that run across the United States:
In the social sciences, nothing ever dies; it merely becomes embarrassing. We can’t say that it’s impossible for anything like these programs to ever succeed, given enough time, flimsy theorizing, and generous definitions of “success”. The point is that these effects should be large, and instead they are nonexistent. Seems to me like it’s time to get the goats in here.
It’s inevitable that some of our interventions are going to fail. (If they don’t, we should either be suspicious of the results, or we should be more ambitious with our programs.) What we really want to know is: when we try to improve the world through some policy or program, how often do we succeed?
We don’t have a good answer to this question, but we do have a good joke about it.
In 1987, the sociologist Peter Rossi wrote a tongue-in-cheek paper that accidentally became his intellectual legacy. He would later refer to it as “far from being one of my favorites” and “the source of considerable embarrassment”, and yet it’s the only thing of his that I’ve ever read or even heard of.3 It was called “The Iron Law of Evaluation and Other Metallic Rules“, and it went like this, lightly paraphrased:
The Iron Law of Evaluation: The net expected impact of a large social program is zero.
The Stainless Steel Law: Better-designed evaluations are more likely to find null effects.
The Brass Law: The more a program is intended to change people, the more likely the impact of that program will be zero.
The Zinc Law: The programs most likely to be evaluated are those most likely to fail.
Rossi tried to walk back his laws shortly before he died in 2006, insisting that there is now a “large number of well conducted impact assessments that yield statistically and substantively significant effect sizes”. Still, even after supposedly enduring 16 years of embarrassment from his one satirical paper, he concluded, “we really do not know whether programs over a wide variety of substantive areas are worth anything”.
Rossi did not do us the favor of actually naming any of those “statistically and substantively significant effect sizes”, and I know why, because I have gone looking for them, and they are not easy to find. The Earned Income Tax Credit, which is designed to benefit low- and middle-income families, was once considered a slam dunk; now a new analysis suggests that it might not do anything. Plain packaging—where you strip cigarette boxes of their attractive branding—seems like a no brainer, but in Australia, it looks like the policy caused smokers to switch to cheaper brands and to smoke more. There’s a reading and math program that has done well in India, but as soon as other researchers tried to implement it elsewhere, the effects shrank.
You can always find a one-off study showing that an intervention works, but if we’ve learned anything from the past 20 years of social science, it’s that one-off studies ain’t worth much. It’s so easy to juice your stats and journals are so hungry for positive results that entire literatures can be suspect, and decades of research can culminate in a shrug. “Sometimes it works and sometimes it doesn’t, it depends on all sorts of factors that we can’t really specify, so uh, good luck!”
Even if we take Rossi at his word and there are as-yet-unnamed programs out there that are reliable and effective, the question still remains: why aren’t there more of them? Why do so many of our best-laid plans end up like this:
OJ A-OK
The answer lies, I think, in a 50-year-old study that is almost certainly false.
One of my favorite genres of psychology experiment is called something like: “the study is probably bullshit but the effect is probably real”. At the top of that heap is The Illusion of Control, a classic from 1975 and a favorite of Psych 101 classes everywhere. Usually, this study gets summarized as “people value their lottery tickets more when you let them pick the numbers vs. when you pick the numbers for them.” It’s as if people think that the mere act of picking imbues their ticket with a bit of juju, which makes them less willing to part with it.
That would be a beautiful study, but it wasn’t the study that was actually conducted. The “lottery tickets” in question were in fact football cards, each one with a picture of a player, their team, etc. Some participants were given the opportunity to choose their football card and some had a card handed to them. Later, just before the drawing, all of the participants were asked how much it would take to buy their card back. On average, the pickers asked for $8.67, while the not-pickers said $1.86.4
I’m sure you can see the obvious and less exciting explanation for these results: when people were given the chance, they picked their favorite players. The people who didn’t get to pick ended up with a card they liked less. This “illusion of control” effect is merely a “reality of people having preferences and acting on them” effect.5
I wouldn’t let this one go either
Unsurprisingly, well-powered, preregistered replications with actual lotteries have failed to find any difference in picking vs. not-picking. So that’s that: there’s no such thing as the illusion of control.
Except...obviously there’s an illusion of control! Look around! People will blow on their dice at the craps table, they’ll don their lucky undies when the Knicks go to the finals, they’ll scream “look out!” to the hapless protagonists of a horror movie. When I was a kid, my parents used to send me off to play arcade games without any quarters, knowing that I would simply move the joystick in synchrony with the demo playing on the screen, convincing myself that I was causing the characters to move, when in fact I was merely mirroring them.
Put it this way: why do you think The Secret has sold over 30 million copies? It ain’t the beautiful prose or the sterling scientific evidence. It’s because when Rhonda Byrne tells you that “your thoughts are the primary cause of everything,” it feels kind of true. After all, can’t you jinx yourself by thinking the wrong thing? Haven’t you ever been like “whatever you do, don’t spill this soup” and then the next thing you know you’ve got a shirtful of bisque? And if brainwaves are powerful enough to spill soup, who knows what else they might do!
So no, people don’t believe they can magically improve their chances of winning the lottery by picking the numbers themselves. Still, our intentions feel more potent than they should, and this shows up not only in our harmless superstitions, but also in the way we keep meddling in each other’s lives despite having no idea what we’re doing.6
How do I know my preferred policies will have their intended results? Well, because I intend them. That’s not really my answer to the question; it’s the reason I don’t even think of the question in the first place.
Here’s the real evidence of our illusion of control: we’re so certain our interventions will work that we forget to check whether they do at all.
For instance, the National Science Foundation funds over 1,000 STEM education projects, which are aimed at improving curricula, training, student outcomes, etc. According to an analysis by Stuart Buck, 76% of the time, the researchers running those programs don’t even attempt to evaluate them with actual experiments. Apparently if you have a bright idea about how to teach kindergarteners to code, you don’t need to bother assessing it. It turns out George W. Bush was actually right when he said:
Speaking of the government, according to a report from the Government Accountability Office, many federal programs don’t even define their goals, let alone collect data on their progress toward those goals. For instance, the Department of Homeland Security is supposed to stop people from producing and distributing illicit fentanyl, and so they do things like search people’s cars and walk around with drug-sniffing dogs. When DHS was asked, “How do we know any of this is working? What would success even look like?”, they were like “hmm great questions”. They hadn’t set any goals or collected any data, despite being required to do so by law. As the GAO puts it with characteristic mildness: “As a result, DHS’s ability to fully understand the effectiveness of these efforts is limited.”
Maybe it’s too much to expect that the Department of Homeland Security, especially in its current incarnation, would try to figure out whether it’s doing things that actually work. But it’s certainly not too much to expect of private philanthropists, who choose to give away their money of their own free will. Certainly they would care whether their dollars do anything. Right?
Wrong. In 2018, Kenneth Langone, the Home Depot billionaire, donated $100 million to eliminate tuition fees at New York University’s main medical school. This was suppose
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。