After giving two examples (Zoltar, Coin Flipping), it is now time in Uncertainty Wednesday to formalize the ideas. The following picture will be useful. It shows a very simple reality/system (the blue box) we are trying to understand and the observation (the arrow) we are making to do so.
In particular, our system has only two possible states A and B. And our observation can take on only two possible values, 0 and 1.
This is the simplest possible setup that is of interest.
Why? Let’s see if it makes sense to go to a reality/system with only one state. Well, in that case what more could possibly be learned? Having only one state, is the same thing as saying that we have a perfect explanation that leaves *no* room for uncertainty! The two states also need to be mutually exclusive (at least for now to keep things classical – meaning as opposed to quantum). For example: either the planets revolve around the Sun, or the Sun and other planets revolve around the Earth (our solar system is either heliocentric or geocentric). Or take a small system with a valve which can be either open or closed. That’s an example where the system can go back and forth between the two states (open, closed) but they are mutually exclusive.
What about our observation? Could we have an observation with only one value? A constant valued observation provides *no* signal. There is nothing that can be learned from it. Imagine the fuel gauge in your car having a single value. It would tell you nothing about the fuel level in your gas tank. Now I should be quick to point out this is not the same as having an observation that has been constant so far but that *could* change in the future. I am not saying we already have to have received a signal, just that whatever it is we are observing has to have multiple *possible* values. Otherwise it will be pointless. For observations two we will assume that values that can be observed are mutually exclusive.
So we can now combine the two system states and the two observation values into 4 possible combinations
State A, Observation 0
State A, Observation 1
State B, Observation 0
State B, Observation 1
Let’s start with a really simple case. Let’s say that *always* when the system is in state A, we observe 0 and when the system is in state B we observe 1. Well in that case our observation provides a *perfect* signal. After we have made our observation there is no uncertainty left. When we observe a 0 we now know with certainty that the system is in state A.
When we say today that we are certain the solar system is heliocentric, what we mean is that we have made observations that can *only* arise in a heliocentric state and *never* in a geocentric one. It is not the same as saying we have somehow grasped reality directly. That is simply not possible. All we ever have are explanations and observation.
What is the opposite of an observation that provides a perfect signal? It would be an observation that provides no signal at all. It would be that observing 0 or 1 is *independent* of the state of reality. To give the simplest example of independence. Suppose your fuel gauge in your car is by actually not hooked to your fuel tank at all but to the outside air temperature sensor instead. At this point what you observe on your fuel gauge is independent of how much fuel there is in your fuel tank. You still get different observed values as the outside temperature changes but you learn nothing about how much gas you have from that.
There is one more assumption that we will make. System states are not only Mutually Exclusive, but also Collectively Exhaustive (giving rise to the acronym MECE). Collectively exhaustive means that our explanation doesn’t involve additional states not included in our list. We will assume the same for the values of our observation. Bot of these are strongly simplifying assumptions as they serve to dramatically reduce how much uncertainty is captured by our formalism (one reason why it is dangerous to rely too much on models).
Next week we will get to a definition of probability and apply it to our two state, two value setup.
