This week psychologists, neuroscientists and statisticians have been again prompted to reassess our methods by a widely-circulated preprint (pdf) accusing some in the field of acting like “methodological terrorists”. Thankfully, the initial urge to identify with one or other “side” (and to hit back at “opponents”) has largely given way to more thoughtful, nuanced and long overdue discussions about how research should be conducted. I am not arguing for or against one of the “sides” in this debate, but I am just sharing my perspective.
Scientists have learned that we need to continually question our beliefs about the natural world, and test them against the evidence. As Feynman put it: “The first principle is that you must not fool yourself – and you are the easiest person to fool.” We have to question our beliefs. This applies to our beliefs about the scientific method as well as to our beliefs about the natural world. It applies to ourselves as well as the other people who we think are wrong.