Boulevard of Broken Science
Or the problem of replicability in science.
I believe the scientific method is the one invention that really changed the humanity course. Every other invention since the 17th century happened as a result, so we need to thank our quality of life, health, technology progress and many other great things, for this revolutionary way of developing solutions.
However, we are in 2021 and the trust in science is the lowest it has ever been and many of us just feel lost. Look for anything online and you can find dozens, sometimes thousands of studies that contradict each other. Want to find a study that says smoking is good for your health, I’m sure you can find one or more. Want to find an article about how we should only eat red meat, I’m sure you can find a few dozens. And it goes on. This is such a big problem that there is a wikipedia page for the “Replication Crisis”, a phrase coined in the early 2010s.
In 2016, Nature surveyed 1,576 researchers and more than half agreed there is “a significant crisis” of reproducibility.
Why is this important?
The reproducibility of experimental results is an essential part of the scientific method. If you can’t replicate the results, more often than not, whatever you found out is (probably) not true. And lots of decisions are being made based on studies that either have gross errors in their methodology or cannot be replicated a second time.
“Show me the incentive and I will show you the outcome” — Charlie Munger
Like in everything in life, it all goes down to the incentives. According with James Owen Weatherall, author of The Physics of Wall Street:
“In the sciences, one is mostly incentivized to publish journal articles, and especially to publish the sorts of attention-grabbing and controversial articles that get widely cited and picked up by the popular media. The articles have to appear methodologically sound, but this is generally a lower standard than being completely convincing.”
The problem with incentives doesn’t stop in the publishing of journal articles. The systems of funding and advancement also value novelty and attention grabbing topics, leaving us not with great science, but with questions, doubt and as seen during this pandemic with a population that decides that it knows better.
In 2017, I was already thinking about this topic and I read an article by WIRED Magazine about John Arnold, an American billionaire that was tackling the replication crisis.
One of the projects he founded was the Center for Open Science, an organization whose mission is to increase openness, integrity, and reproducibility of research.
The Center for Open Science developed the Open Science Framework — free, open-source software in which researchers can compile materials, study designs, and data.
“It makes it very easy to make parts or all of that data publicly available, to increase transparency and reproducibility,” says Brian Nosek, founder of the Center for Open Science.
With this article, I’m not proposing solutions. I just don’t think this problem gathers as much attention as it should, even though I’m 100% sure, everyone already dealt, on way or another, with it. There are many problems that need to be solved and I believe this is among the most important. In the coming years, our trust in science will play an even more important role, because:
We need people to believe that Climate Change is real and act to reverse it.
We need solutions that actually work, that can be replicated and which impact can be measured, because science will be crucial in our fight to reverse climate change.
I personally would love to contribute to projects that are tackling this problem. If you know any or have ideas for new projects in this field, please reach out to me.
To receive my articles in your inbox, subscribe to my newsletter.
