13 Conditional probability with Bayes’ theorem

This chapter explains how Bayes’ theorem lets us update beliefs when new evidence appears. Using a simple survey about colds and sneezing, it introduces prior probability, conditional probability, joint probability, and posterior probability, showing how the probability of a cause can be revised after observing an effect. The key idea is that evidence changes confidence in a diagnosis, and Bayes’ theorem provides the exact arithmetic for that update.

The chapter then connects the math to code by representing conditional probabilities as likelihood functions and combining them with priors to form joint distributions. It shows how a likelihood function can model real-world relationships, such as how having a cold affects the chance of sneezing, and how the same machinery can simulate survey data. Because the library uses integer weights, the text also explains how to use the least common multiple to keep the joint distribution exact while preserving the correct proportions.

Finally, the chapter demonstrates how to reason backward from observed effects to likely causes by filtering a joint distribution to obtain a posterior. It applies this idea to medical testing, showing why a rare disease can still have a very low posterior probability even after a highly accurate positive test, and to a coin-flip example where repeated heads shift belief toward a double-headed coin. It ends by broadening the idea to practical uses in software, and by showing that independent random events can be handled with the same joint-distribution tools, including a concise helper for combining distributions and projecting results such as rolling with advantage.

• Our opinions before seeing additional evidence, such as the probability that a random person has a cold, are prior probabilities. We represent them as probability distribution objects in our system.
• Our opinions about conditional probabilities, such as the different probabilities that a person has been sneezing depending on whether they have a cold, are represented as likelihood functions that take a condition and return a distribution.
• Likelihood functions ideally capture facts about the world that are independent of priors. The fact that people with colds sneeze more than those without is independent of the number of people in the world with a cold right now. Likelihood functions are some of the building blocks of statistical models of reality.
• Given a prior and a likelihood function, we can compute the joint distribution: the distribution of every possible pair of conditions.
• From the joint distribution, we can use a combination of a filter and projection to compute the posterior likelihood function. This function takes an observation of an effect, such as sneezing, and returns an improved estimate of a cause, such as having a cold.
• Bayesian probability can be counterintuitive, particularly when a likelihood function represents the accuracy of a test for a rare condition. If false positives are more likely than true positives, seemingly accurate tests can give highly misleading results.
• Programs make decisions about actions to take in a real world full of uncertainty. When making evidence-based decisions in software, make sure you’re applying Bayesian logic. Just because a measurement is 99.5% accurate doesn’t imply 99.5% confidence in the result; the prior must be included in the calculation.

FAQ