The last few Uncertainty Wednesdays had us look at how to model beliefs using probability distributions and then update those with a specific example of using the beta distribution. You may have noticed something odd about the way we updated the parameters of the beta distribution: add 1 to α when we observe heads and add 1 to β when we observe tails. This wipes out any and all ordering information. So let’s say you have a total of 100 observations. With this update rule the only thing that matters is the total count of heads and tails respectively. Let’s say that happens to be exactly 50 each, which gives us this beautiful looking distribution:

Which is quite tight around the probability 0.5 for heads.

Yet clearly there is a huge difference between observing some fairly random sequence of heads and tails versus say first 50 heads and then 50 tails, or maybe 5 heads, followed by 5 tails, followed by 5 heads and so on. Say for the last one if we had observed this 20 times in a row, our belief for the next toss – expressed as a distribution – surely shouldn’t look like the picture above. Instead, we would probably put a very high probability on the next toss being heads (the last 5 were tails and so we are due for a reversal).

Why does our current approach not detect that at all? The reason is that embedded in our updating approach was the assumption that the coin tosses were independent of each other. If we wanted to allow for tosses to be influenced by prior tosses (or, more likely, by some underlying system that determines the tosses) we would have to use a more complex initial setup and updating procedure. In a future post I will show how to do this.