Today’s Uncertainty Wednesday will be the concluding post in my mini-series on the problem with p-values. We have already seen that it is much easier than expected to reject a null hypothesis if you have incentives to do so. We also saw that the ability to work backwards and generate hypotheses from the data is a big issue. Today we will consider a more foundational, epistemological problem with p-values: what is it that we are really learning when we are rejecting a null hypothesis? Let’s on...

Today’s Uncertainty Wednesday will be the concluding post in my mini-series on the problem with p-values. We have already seen that it is much easier than expected to reject a null hypothesis if you have incentives to do so. We also saw that the ability to work backwards and generate hypotheses from the data is a big issue. Today we will consider a more foundational, epistemological problem with p-values: what is it that we are really learning when we are rejecting a null hypothesis? Let’s on...

As promised for some time, today in Uncertainty Wednesday, I will talk about p-values and what makes them so problematic. We will once again look at a super simple example by going back to considering a coin flip. As before we will consider the highest uncertainty explanation which is that the mechanism producing the coin flip produces heads (H) and tails (T) each with probability 0.5 and that each flip is independent. Now the idea behind p-values is to attempt an argument of reductio ad absu...

As promised for some time, today in Uncertainty Wednesday, I will talk about p-values and what makes them so problematic. We will once again look at a super simple example by going back to considering a coin flip. As before we will consider the highest uncertainty explanation which is that the mechanism producing the coin flip produces heads (H) and tails (T) each with probability 0.5 and that each flip is independent. Now the idea behind p-values is to attempt an argument of reductio ad absu...

Today’s Uncertainty Wednesday continues our exploration into p-values and why they are problematic. Last Wednesday we saw that if you have incentives to reject a null hypothesis, it takes less work than you would initially think to find data that gets you there. I ended that post suggesting that the problem is even bigger than that. How so? We now live in the age of “big data” – researchers in many fields have access to massive data sets. This lends itself to an approach that has become known...

Today’s Uncertainty Wednesday continues our exploration into p-values and why they are problematic. Last Wednesday we saw that if you have incentives to reject a null hypothesis, it takes less work than you would initially think to find data that gets you there. I ended that post suggesting that the problem is even bigger than that. How so? We now live in the age of “big data” – researchers in many fields have access to massive data sets. This lends itself to an approach that has become known...

Last Uncertainty Wednesday, I introduced the concept of p-values. We looked at the example of a null hypothesis (explanation) that a coin is fair, observing heads (H) or tails (T) six times in a row and rejecting that the coin is fair because the probability of that happening with a fair coin is only 0.03125 which is less than 0.05 (a commonly used cutoff). I ended the post by asking readers to consider the following scenario:You are a researcher who gets paid only if you reject the explanati...

Last Uncertainty Wednesday, I introduced the concept of p-values. We looked at the example of a null hypothesis (explanation) that a coin is fair, observing heads (H) or tails (T) six times in a row and rejecting that the coin is fair because the probability of that happening with a fair coin is only 0.03125 which is less than 0.05 (a commonly used cutoff). I ended the post by asking readers to consider the following scenario:You are a researcher who gets paid only if you reject the explanati...