Today’s Uncertainty Wednesday will be the concluding post in my mini-series on the problem with p-values. We have already seen that it is much easier than expected to reject a null hypothesis if you have incentives to do so. We also saw that the ability to work backwards and generate hypotheses from the data is a big issue. Today we will consider a more foundational, epistemological problem with p-values: what is it that we are really learning when we are rejecting a null hypothesis? Let’s on...

Today’s Uncertainty Wednesday continues our exploration into p-values and why they are problematic. Last Wednesday we saw that if you have incentives to reject a null hypothesis, it takes less work than you would initially think to find data that gets you there. I ended that post suggesting that the problem is even bigger than that. How so? We now live in the age of “big data” – researchers in many fields have access to massive data sets. This lends itself to an approach that has become known...

Today’s Uncertainty Wednesday will be the concluding post in my mini-series on the problem with p-values. We have already seen that it is much easier than expected to reject a null hypothesis if you have incentives to do so. We also saw that the ability to work backwards and generate hypotheses from the data is a big issue. Today we will consider a more foundational, epistemological problem with p-values: what is it that we are really learning when we are rejecting a null hypothesis? Let’s on...

Today’s Uncertainty Wednesday continues our exploration into p-values and why they are problematic. Last Wednesday we saw that if you have incentives to reject a null hypothesis, it takes less work than you would initially think to find data that gets you there. I ended that post suggesting that the problem is even bigger than that. How so? We now live in the age of “big data” – researchers in many fields have access to massive data sets. This lends itself to an approach that has become known...