So last Uncertainty Wednesday, we introduced the simplest possible setup with two states and two signal values. Today we will introduce the concept of probability. If you go on Wikipedia you can find a great many different definitions of probability, such as classical, frequentist, and  Bayesian. If you read all of these you will “probably” find yourself thoroughly confused and feeling you know less afterwards (that’s “certainly” been the case for me).

So I will take a slightly different approach here and at least for now leave out all of the historical ways people have approached this and the many philosophical issues that come up here. So here we go instead:

Probability is how likely something is to happen given the best of our current knowledge.

We will use numbers from 0 to 1 to capture this, with 0 meaning something is impossible (it will never happen) and 1 meaning something is certain (it will always happen). This feels, at least to me, like a fairly intuitive definition with clear bookmarks.

The question now is what kind of additional statements we can make about probability. From last time we had two states A and B and two signal values which I will call H and L today (I used 1 and 0 last time but in retrospect using numeric values may be confusing with the numbers we use for probabilities – this is one of the drawbacks with blogging as I am really writing this one post at a time – apologies to those following along). The critical bit was that both the states A and B and the signal values H and L are mutually exclusive and collectively exhaustive.

So we have four possible combinations: AH, AL, BH and BL. In this simplest of models the world must be in one of these four – since they are all there is. I will call each of AH, AL, BH and BL an elementary event. The set of all elementary events is {AH, AL, BH, BL}. The power set, or set of sets, of the elementary events set then is the following 

{ Ø, {AH}, {AL}, {BH}, {BL}, {AH, AL}, {AH, BH}, {AH, BL}, {BL, BH}, {AL, BL}, {AL, BH}, {AH, AL, BL}, {AH, AL, BH}, {AH, BH, BL}, {AL, BH, BL}, {AH, AL, BH, BL}}

Where Ø denotes the empty set, meaning the set that contains none of the elementary events. This power set contains all possible combinations of elementary events. I will refer to each of these as an event.

So putting that together with our fairly intuitive definition of impossible and certain we can now write:

P(Ø) = 0

and

P({AH, AL, BH, BL}) = 1

where the notation P( ) means probability of, i.e. how likely is this event to happen given what we know. And given how we have set up the problem by definition the world always has to be in exactly one of the elementary events, and each event includes exactly one elementary event *except* for the empty set event. So the empty set event is *impossible* and hence has the probability 0. Conversely the event that includes *all* elementary events *always* happens and hence has the probability 1.

But there is more that we can say about how the probabilities of events have to relate to each other based on how we have set up the problem. We will get to that next Wednesday.