The last few weeks in Uncertainty Wednesday we took a detailed look at the various problems with p-values. Now we will start to work towards an alternative approach based on Bayesian inference. Along the way though we have a chance to think about more foundational ideas, including the subjective versus objective and beliefs versus truths. Today we will start by looking at beliefs and how observations may influence those.

As before with p-values we will use coin flips as they are easy to follow (at least as long as cash hasn’t disappeared entirely). But as we do so, we should keep in mind that this simplicity is actually somewhat deceptive. There is an underlying system that produces the coin flip and the result of heads H or tails T are the resulting signals that we can observe. So when we say something like “I believe this is a fair coin” what we are really saying “I believe that this is a system that will produce an observation of H or T with equal probability.” In fact we are saying even more than that. We are also saying that the history of the system does not matter. No matter what signals we have observed so far, the belief of a “fair coin” means that the next flip could be heads or it could be tails and each with probability 0.5.

Now to start developing an intuition around beliefs and their updates let’s start with an extreme: you belief with certainty that only H will be observed. For instance you might belief the system is “stuck” or in the case of a human flipping a coin you might belief that they chose a coin that simply has heads on both sides! Now you start observing. When you see H, what do you learn? Very little (note: we will eventually see that you have learned nothing). After all, you are observing what you already believe you should observe.

But now imagine that you see tails instead! This should result in a change in belief. You now should belief that T can occur with some probability. Possibly still small, but no longer impossible. Let’s say for example that you observe roughly 1 T for every 99 H. You might be revising your belief to assuming that T is observed with probability 0.01 and H with probability 0.99. You have definitely learned something.

Let’s look at how this is related to the concept of Black Swans. Suppose you believe with certainty that all swans are white. What do you learn from seeing a white swan? Again very little (technically nothing, as we will see). You already believe that all swans are white. But you definitely learn something when you encounter a swan that is not white. Such as when people observed that in Australia there are swans that are black.

This is the same as the idea that the “absence of evidence is not evidence of absence” (a quote often used by Nassim Taleb). Not having observed black swans yet (absence of evidence) does not imply black swans do not exist (evidence of absence). Not having seen tails, does not mean tails can never occur.

To dig deeper into this we will formalize belief mathematically. We will start to do that next Wednesday. In the meantime though you can ask yourself already based on the above: what is a problem with holding extreme beliefs, such as certainty that you will observe H or that all swans are white?