Kim Scheinberg sent me a great article from The Atlantic that relates to my multi-thread rant on epidemiology. Since the article speaks for itself, I’m just quoting points I think are salient. The only words below that are not a direct quote are the headlines (i.e. “Did you know?”). The emphasis is mine as well.
Did you know?
- mammograms, colonoscopies, and PSA tests are far less useful cancer-detection tools than we had been told
- Zoloft, and Paxil were revealed to be no more effective than a placebo for most cases of depression
- staying out of the sun entirely can actually increase cancer risks
- taking fish oil, exercising, and doing puzzles doesn’t really help fend off Alzheimer’s disease
Medicine has caught a plague
we think of the scientific process as being objective, rigorous, and even ruthless in separating out what is true from what we merely wish to be true, but in fact it’s easy to manipulate results, even unintentionally or unconsciously.
There is an intellectual conflict of interest that pressures researchers to find whatever it is that is most likely to get them funded.
49 of the most highly regarded research findings in medicine over the previous 13 years…. 41 percent, had been convincingly shown to be wrong or significantly exaggerated.
… mathematical proof that, assuming modest levels of researcher bias, typically imperfect research techniques, and the well-known tendency to focus on exciting rather than highly plausible theories, researchers will come up with wrong findings most of the time.
“Even when the evidence shows that a particular research idea is wrong, if you have thousands of scientists who have invested their careers in it, they’ll continue to publish papers on it,” he says. “It’s like an epidemic, in the sense that they’re infected with these wrong ideas, and they’re spreading it to other researchers through journals.”
Nature… stated, “Scientists understand that peer review per se provides only a minimal assurance of quality, and that the public conception of peer review as a stamp of authentication is far from the truth.”
…when a research error is outed, it typically persists for years or even decades. He looked at three prominent health studies from the 1980s and 1990s that were each later soundly refuted, and discovered that researchers continued to cite the original results as correct more often than as flawed—in one case for at least 12 years after the results were discredited.
… doctors have all been trained to order these tests…. They’re also trained to ply the patient with whatever drugs might help whack any errant test numbers back into line…. “When you look the papers up, you often find the drugs didn’t even work better than a placebo…. “Just taking the patient off everything can improve their health right away.”
What does this mean for you personally?
How should we choose among these dueling, high-profile nutritional findings? Ioannidis suggests a simple approach: ignore them all.
the odds are that in any large database of many nutritional and health factors, there will be a few apparent connections that are in fact merely flukes, not real health effects…. But even if a study managed to highlight a genuine health connection to some nutrient, you’re unlikely to benefit much from taking more of it, because we consume thousands of nutrients that act together as a sort of network, and changing intake of just one of them is bound to cause ripples throughout the network that are far too complex for these studies to detect, and that may be as likely to harm you as help you. Even if changing that one factor does bring on the claimed improvement, there’s still a good chance that it won’t do you much good in the long run, because these studies rarely go on long enough to track the decades-long course of disease and ultimately death. Instead, they track easily measurable health “markers” such as cholesterol levels, blood pressure, and blood-sugar levels, and meta-experts have shown that changes in these markers often don’t correlate as well with long-term health as we have been led to believe.
If a study somehow avoids every one of these problems and finds a real connection to long-term changes in health, you’re still not guaranteed to benefit, because studies report average results that typically represent a vast range of individual outcomes. Should you be among the lucky minority that stands to benefit, don’t expect a noticeable improvement in your health, because studies usually detect only modest effects that merely tend to whittle your chances of succumbing to a particular disease from small to somewhat smaller.
It’s not just medicine
Medical research is not especially plagued with wrongness. Other meta-research experts have confirmed that similar issues distort research in all fields of science, from physics to economics