Probability

Most probability questions reduce to counting: how many outcomes are there, and how many of them are the ones you want. The rules that follow all rest on that.

Sample spaces

The sample space $S$ is the set of all possible outcomes. When they are equally likely,

$$P(A) = \frac{n(A)}{n(S)}$$

Two fair dice. The sample space has $6 \times 6 = 36$ ordered pairs.

$$P(\text{sum} = 7) = \frac{6}{36} = \frac16 \qquad P(\text{sum} \ge 10) = \frac{3+2+1}{36} = \frac16$$

since 10 arises 3 ways, 11 two ways and 12 one way.

For "at least one", the complement is quicker:

$$P(\text{at least one six}) = 1 - P(\text{no six}) = 1 - \left(\frac56\right)^2 = \frac{11}{36}$$

Counting and probability

When the sample space is too large to list, count it with combinations.

A committee of 3 is chosen at random from 6 men and 4 women.

Total ways: $\binom{10}{3} = 120$.

$$P(\text{all three women}) = \frac{\binom43}{\binom{10}{3}} = \frac{4}{120} = \frac{1}{30}$$

$$P(\text{exactly two women}) = \frac{\binom42\binom61}{\binom{10}{3}} = \frac{6 \times 6}{120} = \frac{3}{10}$$

The addition and multiplication rules

$$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$ $$P(A \cap B) = P(A)\,P(B \mid A)$$

The second holds always. When the events are independent it simplifies to $P(A)P(B)$.

Conditional probability from a table

Two-way tables make conditional probability almost mechanical: the condition selects a row or column, and that becomes the new denominator.

$$P(\text{passed}) = \frac{75}{110} = \frac{15}{22}$$

$$P(\text{passed} \mid \text{female}) = \frac{45}{60} = \frac34$$

$$P(\text{female} \mid \text{passed}) = \frac{45}{75} = \frac35$$

Are passing and being female independent?

$$P(\text{passed}) \times P(\text{female}) = \frac{75}{110} \times \frac{60}{110} = 0.372$$ $$P(\text{passed} \cap \text{female}) = \frac{45}{110} = 0.409$$

These differ, so the events are not independent — women passed at a higher rate than the group as a whole.

Tree diagrams for several stages

A factory has three machines. A makes 50% of output, B 30%, C 20%. Their defect rates are 2%, 3% and 5%.

The first branches give the machine, the second whether the item is defective.

$$P(A \cap D) = 0.5 \times 0.02 = 0.010$$ $$P(B \cap D) = 0.3 \times 0.03 = 0.009$$ $$P(C \cap D) = 0.2 \times 0.05 = 0.010$$

Multiply along a branch; add between branches. The branches leaving any node must sum to 1, and so must the complete set of end-points.

Total probability and Bayes

The law of total probability collects every route to an outcome:

$$P(D) = P(D\mid A)P(A) + P(D\mid B)P(B) + P(D\mid C)P(C)$$

$$= 0.010 + 0.009 + 0.010 = 0.029$$

So 2.9% of output is defective.

Bayes' theorem then reverses the conditioning:

$$P(A \mid D) = \frac{P(D \mid A)P(A)}{P(D)} = \frac{0.010}{0.029} = 0.345$$

A defective item is most likely to have come from machine A — despite A having the lowest defect rate, because it makes half of everything.