Probability Theory

Probability is built on three axioms, and everything else — the addition rule, conditional probability, Bayes' theorem — follows from them. Knowing which rule applies is a matter of reading the question's connective: and, or, or given.

The axioms of probability

For any event $A$ in a sample space $S$:

$$0 \le P(A) \le 1$$ $$P(S) = 1$$ $$P(A \cup B) = P(A) + P(B) \quad \text{when } A \text{ and } B \text{ are mutually exclusive}$$

From these follows the complement rule:

$$P(A') = 1 - P(A)$$

so if $P(A) = 0.3$ then $P(A') = 0.7$.

Venn diagrams and set notation

A Venn diagram is the fastest way to organise the information. Fill in the intersection first, then work outwards by subtraction.

Given $P(A) = 0.5$, $P(B) = 0.4$ and $P(A \cap B) = 0.2$:

• Only $A$: $0.5 - 0.2 = 0.3$
• Only $B$: $0.4 - 0.2 = 0.2$
• Both: $0.2$
• Neither: $1 - (0.3 + 0.2 + 0.2) = 0.3$

The four regions total 1 ✓

The addition rule

$$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$

With the figures above:

$$P(A \cup B) = 0.5 + 0.4 - 0.2 = 0.7$$

The subtraction removes the overlap, which adding $P(A)$ and $P(B)$ counts twice.

Mutually exclusive events cannot both occur, so $P(A \cap B) = 0$ and the rule simplifies to $P(A) + P(B)$.

Conditional probability and independence

$$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$$

Rearranged, this is the multiplication rule:

$$P(A \cap B) = P(B)\,P(A \mid B) = P(A)\,P(B \mid A)$$

With the figures above:

$$P(A \mid B) = \frac{0.2}{0.4} = 0.5$$

Since $P(A \mid B) = 0.5 = P(A)$, knowing that $B$ occurred tells us nothing about $A$: the events are independent. Confirming it the other way, $P(A)P(B) = 0.5 \times 0.4 = 0.2 = P(A \cap B)$ ✓

Tree diagrams

A tree diagram handles several stages, especially without replacement, where the probabilities change as you go.

A bag holds 4 red and 6 blue balls. Two are drawn without replacement.

First branches: $P(R) = \frac{4}{10}$, $P(B) = \frac{6}{10}$.

Second branches depend on the first — after a red is taken, 9 balls remain of which 3 are red.

$$P(RR) = \frac{4}{10} \times \frac{3}{9} = \frac{12}{90} = \frac{2}{15}$$ $$P(RB) = \frac{4}{10} \times \frac{6}{9} = \frac{24}{90} \qquad P(BR) = \frac{6}{10} \times \frac{4}{9} = \frac{24}{90}$$ $$P(BB) = \frac{6}{10} \times \frac{5}{9} = \frac{30}{90} = \frac13$$

Check: $\frac{12 + 24 + 24 + 30}{90} = 1$ ✓

Bayes' theorem

$$P(A \mid B) = \frac{P(B \mid A)\,P(A)}{P(B)}$$

where the denominator is usually built from the law of total probability:

$$P(B) = P(B\mid A)P(A) + P(B\mid A')P(A')$$

Bayes reverses a conditional probability — from "the probability of a positive test given the disease" to "the probability of the disease given a positive test".

A disease affects 1% of a population. The test detects it in 95% of those who have it, and gives a false positive for 4% of those who do not. A person tests positive. What is the probability they have the disease?

$$P(D) = 0.01 \qquad P(+\mid D) = 0.95 \qquad P(+\mid D') = 0.04$$

$$P(+) = (0.95)(0.01) + (0.04)(0.99) = 0.0095 + 0.0396 = 0.0491$$

$$P(D \mid +) = \frac{0.0095}{0.0491} = 0.193 \text{ (3 s.f.)}$$

So fewer than one in five people who test positive actually have the disease — despite a test that sounds highly accurate.