Overview

7 Hierarchical models: Uncovering the layers of data the Bayesian way

Bayesian hierarchical models are introduced as a way to represent data that naturally comes in groups, such as students within schools. Rather than forcing every observation into one shared distribution or treating each group as completely separate, a hierarchical approach acknowledges both common structure and group-specific differences. This makes it especially useful when some groups have limited data, because it can borrow strength across groups while still preserving meaningful variation.

The chapter develops this idea through the classic eight-schools coaching example, where each school has an estimated coaching effect and uncertainty. A fully separate model can overstate uncertainty and exaggerate extreme effects, while a fully pooled model can hide real differences between schools. The hierarchical model resolves this tension by assuming each school’s effect comes from a shared distribution with its own overall mean and spread, letting the data determine how similar or different the groups should be.

This shared structure produces shrinkage: individual group estimates are pulled toward the overall mean, with the amount of pulling controlled by the between-group spread. The result is more conservative and realistic inference, such as more believable estimates of whether coaching helps and which schools perform best. The chapter closes by noting that this logic applies broadly, from sports and web analytics to epidemiology and public health, wherever related groups must be modeled together without pretending they are identical.

The effect of coaching in terms of increase in students’ test scores by the separate model that assumes each school makes up a unique, independent distribution.
The effect of coaching in terms of increase in students’ test scores by the separate model that assumes each school makes up a unique, independent distribution.
The posterior distribution of the effect of coaching under the pooled model. The effect centers around 7.7 points and ranges from −10 to 25.
The posterior distribution of the effect of coaching under the pooled model. The effect centers around 7.7 points and ranges from −10 to 25.
A hierarchical model is a compromise between two extremes: a separate model that doesn’t share information across the subpopulations and a pooled model that assumes there’s no heterogeneity between groups.
A hierarchical model is a compromise between two extremes: a separate model that doesn’t share information across the subpopulations and a pooled model that assumes there’s no heterogeneity between groups.
The structure of the partial pooling model, showing multiple levels of dependence between the variables involved.
The structure of the partial pooling model, showing multiple levels of dependence between the variables involved.
The half-normal distribution assigns probability to only half of the corresponding zero-centered normal distribution 𝒩(0, 𝜎²).
The half-normal distribution assigns probability to only half of the corresponding zero-centered normal distribution 𝒩(0, 𝜎²).
The posterior distribution of the variables in the hierarchical model. The base effect 𝜇 centers around 9, while the spread factor 𝜏 is more likely near 0. Individual effects 𝜃ᵢ are different but mostly resemble the base effect 𝜇.
The posterior distribution of the variables in the hierarchical model. The base effect 𝜇 centers around 9, while the spread factor 𝜏 is more likely near 0. Individual effects 𝜃ᵢ are different but mostly resemble the base effect 𝜇.
Conditional average individual effect 𝐸[𝜃ᵢ | 𝜏] as a function of the spread factor 𝜏.
Conditional average individual effect 𝐸[𝜃ᵢ | 𝜏] as a function of the spread factor 𝜏.
The effect of pooling in shrinking the individual effects towards a global trend.
The effect of pooling in shrinking the individual effects towards a global trend.

Summary

  • Hierarchical models explicitly model observations as arising from different groups that partially share common characteristics but also possess their own unique behavior. This is done by treating the statistics that describe the behavior of each group as samples drawn from a common distribution.
  • Hierarchical models let us pool information across related groups while still allowing each group to have its own parameters.
  • Hierarchical models solve the “all-or-nothing” choice between complete pooling (treating groups as identical) and no pooling (treating groups as totally separate).
  • When the variation across groups is explicitly modeled, we can make inference about the difference in behavior between individual groups conditioned on small or large between-group variation.
  • Posterior inference for hierarchical models automatically balances within-group data and across-group similarity, leading to shrinkage.

FAQ

What is a hierarchical model in Bayesian statistics?A hierarchical model, also called a multilevel model, is a Bayesian model that represents data at multiple levels. Instead of assuming all observations come from one common distribution or giving each group completely independent parameters, it lets group-level parameters come from a shared higher-level distribution.
Why are hierarchical models useful for grouped data like students within schools?They let us model differences between groups while still sharing information across them. This is useful when groups are related, because data from one group can help improve estimates for another group, especially when some groups have little data.
What problem does a separate model create when analyzing each school independently?A separate model ignores information shared across schools. This can lead to overly uncertain estimates and extreme conclusions for schools with few observations, because each school is treated as completely unrelated to the others.
What problem does a fully pooled model create when analyzing all schools together?A pooled model assumes all schools have the same effect, which ignores real differences between schools. This can hide meaningful variation and produce conclusions that are too simplistic or unrealistic.
What does partial pooling mean in a hierarchical model?Partial pooling means the model shares some information across groups, but not all of it. Group-specific estimates are informed by both their own data and the overall pattern across all groups.
What do the parameters μ and τ represent in the eight-schools hierarchical model?μ is the overall or base effect across all schools, while τ controls how much the school-specific effects vary around that base effect. A small τ means the schools are similar; a large τ means they can differ more.
Why is a half-normal prior used for τ?Because τ represents a spread or standard deviation, it cannot be negative. A half-normal prior assigns probability only to nonnegative values while still allowing the model to learn small or large between-group variation.
What is shrinkage in hierarchical models?Shrinkage is the tendency for group-specific estimates to be pulled toward the overall mean. This happens because information is shared across groups, which regularizes extreme estimates and reduces overfitting.
How does the hierarchical model improve the inference for School 1 compared with the separate model?It makes School 1’s estimated coaching effect more conservative by borrowing strength from the other schools. Instead of giving School 1 an overly extreme effect, the model shrinks it toward the overall pattern seen across all schools.
Where else can hierarchical models be applied besides school test scores?They are widely used in real-world settings such as batting averages in baseball, ad clickthrough rates across countries, disease spread across regions, and health outcomes across patient groups. Any situation with related subgroups can benefit from hierarchical modeling.

pro $24.99 per month

  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose one free eBook per month to keep
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime

lite $19.99 per month

  • access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more


Close
choose your plan

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • Grokking Bayes Grokking Bayes ebook for free
Close
choose your plan

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • Grokking Bayes Grokking Bayes ebook for free