Conditional probability & Bayes

New evidence should change your mind — but by how much? Conditional probability is the mathematics of 'given that', and Bayes' theorem is the exact rule for updating beliefs as evidence arrives. It's also where intuition fails most spectacularly.

Build the intuition

Conditional probability: narrowing the world

P(A | B) — 'probability of A given B' — restricts attention to the cases where B happened, then asks how often A also holds. Knowing a card is a face card changes the probability it's a king from 1/13 to 1/3. Conditioning is zooming into the relevant subpopulation; almost all real reasoning is conditional.

P(A \mid B) = \frac{P(A \text{ and } B)}{P(B)}

Joint, marginal, conditional

The joint probability is P(A and B) — both at once. The marginal is P(A) alone, ignoring B (summing over its possibilities). The conditional P(A | B) lives between them. A two-way table makes the trio concrete: cell counts are joint, row/column totals are marginal, and a cell divided by its row total is conditional. Most data analysis is moving between these three views.

Bayes' theorem: flipping the condition

Often you know P(evidence | cause) but want P(cause | evidence) — you know how often a sick person tests positive, but want how likely a positive person is sick. Bayes flips it, and crucially folds in the base rate (prior). When the cause is rare, even a great test produces mostly false alarms — because it's applied to a huge healthy majority. Ignoring the base rate is the single most common probabilistic error.

P(C \mid E) = \frac{P(E \mid C)\,P(C)}{P(E)}

See it move

InteractiveBayes, counted out loud

Imagine 10,000 people. 1% truly have the condition; the test is right 90% of the time for both groups.

True positives

False positives

990

False negatives

True negatives

8,910

Among all positive tests: true vs false share.

Prevalence1%

Test accuracy90%

You tested positive with a 90%-accurate test. Chance you actually have the condition: 8.3% — because positives come from 90 true cases and 990 false alarms from the huge healthy group. The base rate is the silent partner in every test result.

The medical-test paradox in honest counts. Drop the prevalence and watch a 90%-accurate test produce mostly false positives — the base rate is the silent partner.

A worked example

The surprising positive test

A disease affects 1% of people. A test is 99% accurate both ways. You test positive — how worried?
Per 10,000 people: 100 sick (99 test positive), 9,900 healthy (99 false positives).
Positives total 99 + 99 = 198, of which 99 are truly sick:
$P(\text{sick} \mid +) = \frac{99}{198} = 50\%$
A 99% test, yet a coin-flip chance of disease — because the disease is rare. This is Bayes, and almost everyone guesses 99%.

Out in the world

Spam filters think in Bayes

Classic spam filters compute P(spam | words) from P(words | spam) and the base rate of spam via Bayes — the original 'Bayesian filter'. Every 'this seems like spam' decision updates a prior with evidence, exactly as this lesson prescribes.

Common confusion, cleared

“P(A | B) equals P(B | A).”

They're often wildly different: P(positive | sick) ≠ P(sick | positive). Confusing the two — the 'prosecutor's fallacy' — has sent innocent people to prison. Bayes is the bridge between them.

“A test's accuracy is your chance of being sick if positive.”

Only when the disease is common. With a rare condition, the base rate dominates and most positives are false. Accuracy and posterior probability are different numbers.

Check yourself

PracticeQuick check

A rare disease, a very accurate test, a positive result. Your true risk is…

Recap

Conditioning zooms into the subpopulation where the given fact holds.
Joint, marginal, and conditional are three views of the same two-way table.
Bayes flips P(E|C) into P(C|E) — and the base rate decides the answer.

Progress saves in this browser.