From The Conversation: Scientists are increasingly finding that the P value, one of the classic science statistics, is not as reliable as we like to think. Perhaps it is time for scientists to focus on the effect size instead.
“So if not the P value, what should we use to analyse and interpret our data? We argue for a fundamental shift in thinking away from asking the question ‘is there a difference?’ and towards asking ‘how big is the difference?’. After all, scientists rarely want to know simply whether there is a difference between conditions.
There is always a difference, even if extremely small. It is more pertinent to ask whether the difference is big enough to be of interest, to be of importance. If the effectiveness of the red pill is just 0.01% greater than that of the blue pill, there is a difference between them but it isn’t noteworthy – in practice one pill colour is as good as the other.
The P value can be ditched and scientists can focus instead on how big the difference is between the conditions according to their experiment. They can also provide simple-to-calculate values on how precise that difference is likely to be when generalised beyond the laboratory.”
