In a paper forthcoming in Nature Human Behavior, some of the most important figures in the social and behavioral sciences, 72 in all, have issued a call for a broad redefinition of statistical significance. The one sentence summary– “We propose to change the default P-value threshold for statistical significance for claims of new discoveries from 0.05 to 0.005.”
Authors argue that current statistical standards of evidence for new findings are too low, and that this is an important factor in the failure of many studies to replicate which has gained significant attention over the past few years.
Several of the authors recently posted about the work on the Center for Open Science blog. They note that this approach has previously been used (successfully) to improve reproducibility in other domains, pointing to the genetics community as an example. They further point out that there are better approaches to statistical analyses than null hypothesis significance testing. The proposed standard is aimed at research communities that continue to rely on null hypothesis significance testing at a 0.05 threshold, with the hope that it will raise awareness of the limitations of relying so heavily on a P-value threshold and will thereby facilitate a longer-term transition to better approaches.