Moving Beyond the 0.05 p-value
One of my common refrains in research conference and here is that the misuse of p-values has negative public health consequences, a phenomenon I call "death by p-value." Of course my level of frustration with p-values pales in comparison to what well-trained statisticians must feel. This week, the American Statistical Association Board of Directors led by Ronald Wasserstein released a Statement on Statistical Significance and P-values which include six principles on the use and interpretation of p-values. These are:
- P-values can indicate how incompatible the data are with a specified statistical model.
- P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
- Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
- Proper inference requires full reporting and transparency. A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.
- By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.
In addition to the ASA statement I highly recommend the coverage in FiveThirtyEight and Retraction Watch's interview of Professor Wasserstein. We often talk about the post-antibiotic era but even more important for public health is that researchers and journals happily embrace the post p=0.05 era.