2 Priors, likelihoods, and posteriors—What Bayes is all about: Combining intuition and data to update our knowledge
Bayesian reasoning is presented as a natural way to combine what we already believe with what we observe, producing an updated belief called the posterior. The chapter explains Bayes’ theorem as the rule that connects these pieces: a prior captures initial belief, a likelihood measures how well each hypothesis explains the data, and the evidence normalizes the result so all posterior probabilities sum to one. Through simple examples like rain, clouds, and a rare medical test, the chapter shows that Bayesian updating answers a practical question: how should we change our minds when new information arrives?
The chapter then builds intuition for the mechanics of Bayes’ theorem by working through competing hypotheses and showing how priors and likelihoods interact. A hypothesis with a strong prior can still be weakened by poor support from the data, while a hypothesis with a modest prior can become plausible if it fits the evidence well. The text also highlights important edge cases: assigning zero probability to a hypothesis makes it impossible to recover later, and data with zero likelihood rules a hypothesis out entirely. This logic is compared to detective-style elimination, where accumulating evidence narrows the field of suspects.
To make the ideas concrete, the chapter develops a full model for estimating whether people prefer tea or coffee. It defines a discrete set of possible proportions, chooses a prior distribution over those values, and derives the likelihood using the binomial distribution for survey results such as three tea drinkers out of five. Applying Bayes’ theorem yields a posterior that peaks near the observed sample proportion while still expressing uncertainty, especially with small datasets. The chapter closes by showing that larger datasets sharpen the posterior and that reasonable priors usually matter less as evidence grows, as long as they do not rule out plausible answers too early.
Bayes’ theorem and the interpretation of its components.
An example of the Bayesian procedure to update prior probabilities of three hypotheses to their posterior probabilities.
Prior and posterior probability distributions of 𝑝, the proportion of tea drinkers.
Comparison between the prior and posterior distributions of 𝑝 under different datasets. As the size of the dataset grows, the most-likely prediction of 𝑝 becomes more confident.
Comparison between the posterior distributions of 𝑝 under different prior distributions.
Comparison between the posterior distributions of 𝑝 under different prior distributions and a large sample size. Most prior distributions converge to similar-looking posteriors under a lot of data.
Summary
- Bayesian probability models our belief about an unknown quantity using a set of competing hypotheses, each corresponding to a value that the quantity of interest can take. The probability of a hypothesis quantifies how likely we believe it is for the quantity to take that value.
- Bayes’ theorem states that the posterior probability is proportional to the product of the corresponding prior and likelihood.
- Multiplying the corresponding prior and likelihood yields the unnormalized posterior probability. We normalize these probabilities so that they sum to 1 to obtain the true posterior probabilities.
- The Bayesian update rule models a detective’s search for a culprit of a crime: the detective updates the probability that each suspect committed the crime based on the evidence that the detective uncovers.
- The choice of the prior distribution in a Bayesian model reflects your belief about the quantity of interest and is subjective.
- The likelihood is the probability of the data under each hypothesis and thus quantifies how well a hypothesis explains the data.
- The posterior distribution in a Bayesian model reflects both the prior and the data.
- A hypothesis that has a zero prior probability (when we rule out the hypothesis initially) or a zero likelihood (when the data rules out the hypothesis) is guaranteed to have a zero posterior probability.
- Under a lot of data, many priors lead to similar posteriors.
Grokking Bayes ebook for free
