Isegoria

In Lies, Damned Lies, and Medical Science, David Freedman looks at the work of meta-researcher John Ioannidis:

He chose to publish one paper, fittingly, in the online journal PLoS Medicine, which is committed to running any methodologically sound article without regard to how “interesting” the results may be. In the paper, Ioannidis laid out a detailed mathematical proof that, assuming modest levels of researcher bias, typically imperfect research techniques, and the well-known tendency to focus on exciting rather than highly plausible theories, researchers will come up with wrong findings most of the time. Simply put, if you’re attracted to ideas that have a good chance of being wrong, and if you’re motivated to prove them right, and if you have a little wiggle room in how you assemble the evidence, you’ll probably succeed in proving wrong theories right. His model predicted, in different fields of medical research, rates of wrongness roughly corresponding to the observed rates at which findings were later convincingly refuted: 80 percent of non-randomized studies (by far the most common type) turn out to be wrong, as do 25 percent of supposedly gold-standard randomized trials, and as much as 10 percent of the platinum-standard large randomized trials. The article spelled out his belief that researchers were frequently manipulating data analyses, chasing career-advancing findings rather than good science, and even using the peer-review process—in which journals ask researchers to help decide which studies to publish—to suppress opposing views. “You can question some of the details of John’s calculations, but it’s hard to argue that the essential ideas aren’t absolutely correct,” says Doug Altman, an Oxford University researcher who directs the Centre for Statistics in Medicine.

Still, Ioannidis anticipated that the community might shrug off his findings: sure, a lot of dubious research makes it into journals, but we researchers and physicians know to ignore it and focus on the good stuff, so what’s the big deal? The other paper headed off that claim. He zoomed in on 49 of the most highly regarded research findings in medicine over the previous 13 years, as judged by the science community’s two standard measures: the papers had appeared in the journals most widely cited in research articles, and the 49 articles themselves were the most widely cited articles in these journals. These were articles that helped lead to the widespread popularity of treatments such as the use of hormone-replacement therapy for menopausal women, vitamin E to reduce the risk of heart disease, coronary stents to ward off heart attacks, and daily low-dose aspirin to control blood pressure and prevent heart attacks and strokes. Ioannidis was putting his contentions to the test not against run-of-the-mill research, or even merely well-accepted research, but against the absolute tip of the research pyramid. Of the 49 articles, 45 claimed to have uncovered effective interventions. Thirty-four of these claims had been retested, and 14 of these, or 41 percent, had been convincingly shown to be wrong or significantly exaggerated. If between a third and a half of the most acclaimed research in medicine was proving untrustworthy, the scope and impact of the problem were undeniable. That article was published in the Journal of the American Medical Association.

Steven Pinker gave a nine-minute speech for the Foundation for Individual Rights in Education on the topic of Taboo:

Thank you very much. It’s a great honor to be speaking to the Ford Hall Forum. All the more so in an event designed to honor the Foundation for Individual Rights in Education.

I’m an experimental psychologist. I’m interested in how the mind works, and that gives me two interests in the issue of free speech on campus. One of them is that I think that psychology is becoming the science that is most likely to get people exercised about issues that they feel have great moral and cosmological import. It used to be that people were deeply concerned about cosmology, whether the earth went around the sun or vice versa, and read profound moral implications into that debate. That pretty much got settled. Then, I think it was evolutionary biology. The battle is not quite won, but clearly, among educated people, it’s clear that that’s no longer an open issue, whether life evolved by Darwin’s mechanism of natural selection.

Today, I think it is the scientific study of the mind that people tend to blend with deep moral issues. I’ll give you just a few examples of questions that have been raised by people in the field of psychology that have gotten them into trouble because even though they, in theory, are purely intellectual questions, people believe that they shake the foundations of morality.

Do most victims of sexual abuse suffer no lifelong damage? Do women, on average, have a different average aptitude in mathematical reasoning than men? Are Ashkenazi Jews on average smarter than Gentiles because their ancestors had been selected for the shrewdness needed in money lending? Is morality just a gadget that evolution installed in our brains with no inherent reality? Are religious beliefs like parasites, which colonize the minds of believers? Is the average intelligence of Western nations falling because duller people are having more children than smarter people? Do men have an innate tendency to rape? Do women who give birth under difficult circumstances have an innate tendency to abandon or even kill their newborns?

Now, if you feel your blood pressure rising as you listen to this list of research questions, then you will have first-hand acquaintance with the fact that scientific or empirical questions can have moral colorings, or at least they can be perceived to have moral colorings. The people who have raised these questions have often been persecuted in ways that the Foundation for Individual Rights in Education is well aware of.

But as a psychologist, I have a second and deeper interest in this set of issues, which is, why does our blood pressure rise? Why did the hairs on the back of our neck stand up when we entertain purely intellectual research questions such as these?

It brings up a phenomenon called the Psychology of Taboo, the sensation that certain ideas are evil to think. Quite apart from the fact, of course, that certain actions are evil to commit, but can it be sinful even to think a thought? Now, most of us would consider ourselves above this primitive mindset. The word taboo comes from animistic beliefs in Polynesian preliterate peoples, and most of us consider ourselves above that.

If you do, I’m going to invite you to entertain the following questions in social gatherings. So when you get home tonight, you and your spouse can have the following conversation: Darling, I know that you always have been faithful and always will be faithful, but just hypothetically, if you were going to have an affair, which of my friends would it be with? Or try this: In a dinner party, ideally one of mixed ethnicity and religion, ask the question, well, of course none of us here are prejudiced, but if we were, if you were bigoted, which ethnic group would you be prejudiced against, just hypothetically? How much money would you accept to betray your best friend, to spread confidential information about him or her? Let’s say your aged parent needed an expensive medical procedure that would prolong his or her life by two months, or six months, how many thousands of dollars would you be willing to relinquish in order to grant your beloved parent those extra two months of life?

Or, a couple of examples that have been the focus of dramatic plots: Let’s say the Nazis forced you to give up one of your children, and if you refused to decide they would kill both of them, which child would you give up; of course, the plot of Sophie’s Choice. Or, if you were in financial dire straits and a wealthy man offered you a million dollars to sleep with your wife, or with you if you are a woman, would you accept? Now, there’s a joke about that — that’s of course, the plot of Indecent Proposal — where a man walking out of a movie theater with his wife said “would you sleep with Robert Redford for a million dollars?” She said, “well, yes, but they would have to give me some time to come up with the money.”

Now, with these questions, the longer you think about them, the more you incriminate yourself. The answer to these questions is not to deliberate and then say, well, no, I wouldn’t want to sleep with any of your friends, or now that I think of it, there isn’t any ethnic group that I’m prejudiced against. One must reject them instantly. Just allowing them to percolate in your brain is considered morally corrosive. As a psychologist, I can recognize the function of this quirk of our psychology, namely, as we associate with other people, as we commit ourselves to our friends and our family, we care not just about what people think or what they do, but what kind of person they are.

It’s one thing if your friend or your spouse has been good to you so far, but are they always eyeing a possible better deal? Would they stab you in the back or sell you up the river as soon as the circumstances made it profitable for them? We don’t want to have to constantly ask those questions, and so we seek life partners, coalition partners, friends who are committed through and through, who would not even consider betraying us because it runs against every fiber of their makeup. That is why there is such a thing as the Psychology of Taboo, and all of those questions, which theoretically should be innocent, in fact are corrosive because they require people to think exactly the kind of thoughts that they should not think if they are committed friends, allies, family members.

Well, it makes a lot of sense from the point of view of our psychology, but now, along comes this institution called academia, or for that matter, journalism, government, the judicial system, which is, at least, nominally devoted to pursuing the truth no matter how uncomfortable it makes people emotionally. Hence, we have the often emotional reactions to purely intellectual questions in all of these spheres of activity, and the dilemma of whether in these truth-seeking organizations, the right of inquiry should be absolute. That is, should we dismiss our taboo reactions as a primitive part of our psychology, which would just get in the way of the mission of these modern institutions of truth-seeking, or do they deserve some respect that carries over to these formal spheres?

I don’t know the answer to that question in every case, but I do know that left to its own devices, human nature will be more outraged, more likely to censor, more likely to be victim to the Psychology of Taboo than would be optimal for the progress of human understanding. It’s for that reason that I’m grateful that we have an organization like FIRE to push back against the tendency in human nature to squelch inquiry under the mentality of taboo. Thanks very much.

(Hat tip to Aretae.)