Bayes' rule
Bayes’ rule expresses the probability of a hypothesis given positive-probability evidence as their joint probability divided by the evidence probability.
On this page 8 sections
01Bayes’ rule updates a belief about a hypothesis after observing evidence#
Bayes’ rule updates a belief about a hypothesis after observing evidence. It connects three quantities: how common the hypothesis was beforehand, how likely the evidence was under it, and how common that evidence is overall. These are different probabilities with different denominators.
Bayes’ rule reverses a conditional. For a hypothesis H and evidence E, the chance of E within H need not equal the chance of H within E.
Suppose P(H)=0.3 and P(E | H)=0.8. Their joint mass is 0.24. If all evidence has mass 0.4, H accounts for 0.24 / 0.4 = 0.6 of it.
In 1000 records, suppose 100 satisfy a hypothesis and 90 of those show the evidence. If 300 records show the evidence in total, then 90 of the 300 evidence records satisfy the hypothesis. The posterior is . The 90% evidence rate within the hypothesis does not reverse to a 90% hypothesis rate within the evidence.
For :
The prior is P(H). The likelihood is P(E | H). Divide their joint mass by positive total evidence P(E) to obtain the posterior.
P(H)=0.4, P(E | H)=0.5, P(E)=0.25. Find P(H | E).
Show answer and explanation
0.2 / 0.25 = 0.8.
02The same reversal works with counts#
The same reversal works with counts. If 40 records have H, half of them show E, and 80 records show E overall, the posterior is 20 / 80 = 0.25.
Among 200 uniform records, 50 have H. Within H, the E rate is 0.4. Exactly 40 records have E. Find P(H | E).
Show answer and explanation
20 H-and-E records out of 40: 0.5.
03All ways of getting E belong in the denominator, including evidence without H#
All ways of getting E belong in the denominator, including evidence without H. A zero likelihood can give posterior 0; zero total evidence makes this ratio undefined.
P(H)=0.25, P(E | H)=0.8, P(E)=0.5. “P(H | E)=0.8.” Repair it.
Show answer and explanation
Joint 0.2; divide by 0.5: 0.4.
04A closer look#
In a model that updates candidate explanations after a new observation, Bayes’ rule preserves both prior plausibility and evidence fit. Include all routes to the observed evidence in its total probability. Strong-looking evidence can still leave a modest posterior when the hypothesis is initially uncommon.
Multiply prior by likelihood, then divide by all evidence. The two conditional directions use different denominators.
- Reverse a conditional using the full evidence total.
Sources & further reading
- [1]Pishro-Nik, Introduction to Probability: 1.4.3 Bayes Rule ↗Pishro-Nik, Introduction to Probability · Book