Skeptical science: Questionable data plagues scientific research

Those who think science is the measure of all truth might want to check the data first.

Here’s a quote for you: “A lot of what is published [in scientific journals] is incorrect.” Care to guess where those words appeared? Not on a website that questions the “consensus of experts on climate change.” Nor do they appear in a publication associated with intelligent design or other critiques of Neo-Darwinism.

They appeared in the April 11, 2015, issue of the Lancet, the prestigious British medical journal.

The writer, Richard Horton, was quoting a participant at a recent symposium on the “reproducibility and reliability of biomedical research.” Specifically, the symposium discussed one of the “most sensitive issues in science today: the idea that something has gone fundamentally wrong with one of our greatest human creations.”

And he’s referring to scientific research—the research that not only purports to tell us how the world works, but, increasingly, how people should order their lives and societies.

As Horton told Lancet readers, “The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.”

He continues, “In their quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world.”

We recently saw an example of this in a story about a much-publicized study purporting to show that voters were likely to change their minds about same-sex marriage if they were visited by gay pollsters who shared their stories with them.

Researchers seeking to reproduce the findings found discrepancies in the data and asked the original researcher for the original data. The researcher was unable to produce the original data. This led the lead researcher to request that the study be withdrawn. Even supporters of same-sex marriage acknowledged that the study and the conclusions drawn from it were fraudulent.

While this was the most-publicized example of a study that turned out to be untrue, it’s far from the only one. In late April, researchers published the results of their efforts to replicate 100 of “psychology’s biggest experiments.” They were only able get the same results in 39 of them.

Commenting on the failure, Daniele Fanelli of Stanford told the prestigious journal “Nature” that “reproducibility rates in cancer biology and drug discovery could be even lower.” She added, “From my expectations, these are not bad at all.”

Not bad at all?

It’s true that to err is human, but insisting that science is the best and most reliable guide to life despite its checkered track record is more than error. It’s chutzpah.

According to Horton, the participants at the symposium he attended agreed that “something needed to be done” about the “bad scientific practices” he describes, but none of them were sure about what that “something” might be.

Well, I have a suggestion: stop telling everyone else that science is the best, if not only, way to answer life’s big questions. While this advice may not fix the problems in the lab, it keeps the “turn toward darkness” from spreading misinformation to the rest of society.

One more suggestion: if a lot of the stuff being published is “incorrect” or “untrue,” please refrain from comparing people who question the scientific consensus to Holocaust deniers and flat-earthers.

A little bit of humility would not be bad at all.

Eric Metaxas

— by Eric Metaxas

Metaxas is currently the voice of Breakpoint, a radio commentary ( Copyright© 2015 Prison Fellowship Ministries. Reprinted with permission. BreakPoint is a ministry of Prison Fellowship Ministries.

Don't Miss Out!

Subscribe to the CNJ newsletter for the latest breaking news, commentary, entertainment,  contests, and more!