6 Coincidence? I think not!

Hypothesis testing is presented as a way to challenge a claim using data, with a null hypothesis representing the status quo and an alternative hypothesis representing a competing explanation. The chapter introduces this idea through a memorable tea experiment, where the question is not whether something is absolutely true, but whether the observed result would be so unlikely under random chance that the null hypothesis becomes hard to defend. The central takeaway is that statistics works by weighing evidence, not proving certainty, and that strong skepticism, humility, and careful experimental design matter.

The chapter then builds the mechanics of two-tailed testing for sample means, showing how to compare a sample to an assumed population value using a test statistic, a confidence level, and a significance level. It explains that the significance level is the leftover tail area outside the confidence interval, and that a result is rejected when the test statistic lands in that extreme region. A potato chip example demonstrates the process end to end: a sample mean that is far enough from the advertised mean produces a test statistic outside the critical range, leading to rejection of the null hypothesis.

Finally, the chapter explains how p-values summarize the same decision rule more conveniently by measuring how extreme the observed sample would be if the null were true. It contrasts two-tailed and one-tailed tests, showing that one-tailed tests place all the significance area in one direction and are therefore less conservative, but only appropriate when the direction of the claim is known in advance. The chapter closes by extending the same logic to smaller samples using the t-distribution and to proportions and independent-sample comparisons, reinforcing that hypothesis testing is a flexible framework for deciding whether observed differences are likely meaningful or just due to chance.

A critical Z-value of ±1.959964 for a 95% level of confidence

A critical Z-value of ±1.959964 for a .05 level of significance (alpha)

The p-value is the probability of values more extreme than your sample.

Comparing the p-value to α. Since the p-value is less than α, we reject the null hypothesis.

Showing the one-tailed test confidence and significance levels in the left direction

Showing the one-tailed test confidence and significance levels in the right direction

If our test Z-value falls into the significant region, we reject H0. Otherwise, we fail to reject H0.

A one-tailed test makes it easier to reject H0 because the tail makes a bigger target to hit.

Zooming in on the left tail, the p-value for our left-tailed test is compared with α = .05. Because the area of p is less than α, we reject the null hypothesis.

• Hypothesis testing uses sample data to make inferences about a population parameter, often to test a claim or theory.
• The null hypothesis (H₀) represents the status quo or no-effect assumption (e.g., “the population mean is 100”), which we assume to be true unless strong evidence suggests otherwise.
• The alternative hypothesis (H_a) states what we suspect might be true instead (for example, “the population mean differs from 100”), and we use sample data to decide whether there’s enough evidence to reject H₀ in favor of H_a.
• Two-tailed tests check for any difference from the hypothesized value (H_a: parameter ≠ value), splitting the significance level (α) across both tails of the distribution.
• One-tailed tests are directional: left-tailed for “less than” (H_a: parameter < value) or right-tailed for “greater than” (H_a: parameter > value), with α concentrated in one tail.
• To reject H₀, the test statistic (calculated from the sample) must fall in the critical region (defined by α, such as 0.05 for 5% significance), or the p-value must be less than α, indicating that the sample result is unlikely under H₀.
• For tests on means, use the standard normal distribution if the sample size n > 30; prefer the t-distribution (with df = n – 1) for n ≤ 30. Many practitioners use the t-distribution in all cases to be safe.

FAQ