Today in Uncertainty Wednesday we are continuing with some of the math of probability. As a reminder from last Wednesday, we have two states, A and B and two signal values H and L for four possible elementary events AH, AL, BH, and BL. We defined events as some combination of elementary events. So for instance we could have E = {AH, BL}. We write P(E) for the probably of event E and we saw that P(Ø) = 0 and P({AH, AL, BH, BL}) = 1.

Since events are sets, we can also reason about the probability of different events to make sure it is consistent with our notion that “Probability is how likely something is to happen.”

Assume E ⊂ F, meaning the event E is a subset of the event F, i.e. every elementary event that is in E is also in F (and there may be some more in F), then we know that

P(F) ≥ P(E)

This is an important monotonicity condition. As you add more elementary events to an event, the probability of that event can only go up. This is related to another intuitive condition, which is that for any event E

P(E) ≥ 0

That is the probability of any event can never be negative (but it can be zero).

Because elementary events are mutually exclusive *by definition* we can also figure out how to add probabilities. Take two events E and G, such that

E ∩ G = Ø

i.e. E and G are mutually exclusive, which means that there are no elementary events that belong to both E and G, then

P(E ∪ G) = P(E) + P(G)

Or in words, the probability of the combined event which contains the elementary events from E and G, is simply the sum of the individual probabilities.

It turns out that you can pick three of these observations as probability axioms and then use that to derive the others. We won’t pursue this axiomatic approach in detail here as I am not sure it adds much to the level of understanding I am shooting for.

Speaking of level of understanding, I should be quick to point out that in practice without thinking about them in a mathematical formulation, humans are bad even at these basic probability statements. Here is just one example of how our built in intuitions or heuristics often go against this math. This is known as the conjunction fallacy:

Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.

Which is more probable?

1. Linda is a bank teller.

2. Linda is a bank teller and is active in the feminist movement.

A surprising (?) number of people will choose (2) as the answer. Our brain takes the story elements and maps them to a mental picture of Linda. But in our mathematical formulation 2 ⊂ 1, meaning the event 2 (in which Linda is both a bank teller and active in the feminist movement) is a subset of 1 and so it must be that P(2) ≤ P(1).