Today's blog is from Dan Harvie. Dan is PhD student from the Body in Mind research group in Adelaide, Australia.
Why science is not necessarily self-correcting
As disciples of science we trust in data, worship the shrine of a computer screen displaying an SPSS spread-sheet and ‘praise the Lord’ when our prophetic hypotheses are blessed with a p-value of less than 0.05.
We are schooled to believe that even when truth is distorted (through chance, bias or false witness), that the ‘self-correcting’ nature of science will see fallacies exposed and truth restored.
This redemption of truth comes about through scientists who heed the call to serve, through replication. In the words of John P. A. Iannidis (1):
"Without replication all we have is ‘unconfirmed discoveries’ and ‘unchallenged fallacies."
Through replication, both false-positive and false-negative findings are revealed. In reality however, replication rarely occurs and calling science ‘self-correcting’ might be an unfounded conviction. In his paper ‘Why Science is Not Necessarily Self-Correcting’, John quotes some facts1 (probably based on ‘non-replicated’ research) which cast doubt on our faith in science:
-Only 1-5 % of positive findings are replicated.
-A little over half of these are conceptual rather than direct replications (i.e. did not replicate ‘exactly’).
-Furthermore, when research is repeated the reproducibility of results is dismal.
These are some of the barriers to self-correcting science(1):
-Selective reporting bias
-Fabricated results and other questionable research practices
-No replication work done and underestimation of this as a ‘crisis’
-Editorial bias against replication research
-Reviewer bias against replication research
-Data, analyses, protocols not publicly available
How can we restore faith in Jesus Science?
Science will never be self-correcting unless direct (not just conceptual) replications are performed. Whether a replication study yields the same or a different result, the outcome is favourable to the credibility of scientific evidence and the only non-bias goal of research: “the pursuit of truth”. In order to improve the trustworthiness of science John Iannidis recommends that we1:
-Publish null results
-Embrace direct replications
-Conduct rigorous research
-Report methodology well
1. Ioannidis JP: Why science is not necessarily self-correcting. Perspectives on Psychological Science 7:645-54.