In the last two installments of Uncertainty Wednesday we looked at the concept of independence and concluded that it was actually quite a strong assumption. Now as one of the comments pointed out, independence is normally introduced in the context of a repeated coin flipping, where it is generally assumed that the outcome of the next coin flip is independent of the prior coin flip. When the independence assumption holds it means that no matter how long a sequence of coin flips I give you, you can do no better with your prediction for the next flip than saying 50% heads and 50% tails.

Now we had previously looked at coin flipping through the lens of our framework. At the time I had pointed out that a coin flip is a signal about the machinery producing the coin flip. So now we can restate the independence assumption about coin flips as follows:

No matter how many coin flips you observe, you cannot learn anything useful about the mechanism producing the coin flips (useful meaning something that can be used to better predict the next coin flip).

Again, it is important to let this one sink in and make sure you really understand what is going on. The best way to do this is by focusing on the fact that a coin flip is a deterministic mechanical process without quantum randomness. If you throw a coin into the air with precisely the same vertical speed and rotational frequency you will get precisely the same outcome.

So the randomness in the outcome of the coin flip comes from changes in the vertical speed and rotation. But those changes in turn must be produced by the system flipping the coin. So why is it that we cannot learn anything about that system?

The answer is that there are a number of reasons for this. First, the result of a flip is an incredibly compressed signal. It has just two values: heads or tails. We had early on identified resolution as a limit on observations. A coin is a very low resolution signal. Now if the system producing the coin flip is a mechanical machine with just a few possible states, then that’s useful. But if a human is flipping the coin, we can see how the human has a huge number of internal states and expecting to learn much or anything about them from such a narrow signal is hopeless.

Second, even if we are just trying to figure out what the vertical speed and rotational frequency of the flip were that produced the current outcome, it turns out that the relationship between the two is highly non-linear. Meaning small changes in either will have the signal “flip” from heads to tails or vice versa. So while there is a deterministic relationship producing the signal, the map of that relationship even to just two variables is incredibly complicated. Here is a beautiful video by the mathematician Persi Diaconis which illustrates this relationship (hat tip to my friend Volkmar for pointing me to this)

So what should you take away from this? Assuming independence on coin flips leads one down a path of lots of combinatorial analysis about probability. That’s how it is taught early on in most courses on probability and we may look at that in a later post. But that analysis obscures the much more interesting question, which we have looked at here: Why is the independence assumption a reasonably good one for coin flips?