Fixed Effects vs Random Effects and the Hausman Test

With panel data (the same units, such as people, firms, schools, or countries, observed repeatedly over time), the main modeling choice is between a fixed effects model and a random effects model. Fixed effects uses only the variation within each unit over time, which removes all stable, unmeasured differences between units. Random effects also uses variation between units, which is more efficient but only valid if those unmeasured differences are unrelated to your predictors. The Hausman test is the standard way to check whether that assumption is plausible.

This guide explains both models, the logic of the Hausman test, a worked example, and how to report the results.

The panel data problem

Suppose you study whether R&D spending raises firm productivity using 200 firms observed for 5 years. Firms differ in ways you cannot measure: management quality, culture, location advantages. If better-managed firms both spend more on R&D and are more productive, a simple pooled regression will attribute part of the management effect to R&D. That is omitted-variable bias.

Panel models write the outcome as:

y_it = β x_it + α_i + ε_it

where i indexes units, t indexes time, and α_i is the unit effect: everything stable about unit i that affects the outcome. The two models differ in how they treat α_i.

The fixed effects model and the within transformation

The fixed effects (FE) model treats each α_i as a separate parameter to be removed. The usual way is the within transformation: subtract each unit's own mean from every variable.

(y_it − ȳ_i) = β (x_it − x̄_i) + (ε_it − ε̄_i)

Because α_i is the same in every period for unit i, it disappears when you subtract the unit mean. The model now asks: when a unit's x is above its own average, is its y also above its own average? Estimating this by OLS gives the same β as including a dummy variable for every unit (the least squares dummy variable, or LSDV, approach).

A tiny illustration

Two firms, three years each:

Within each firm, a one-unit increase in x goes with a one-unit increase in y. But firm B spends more and is less productive overall. A pooled OLS regression that ignores the firm gives a slope of −0.71. After demeaning (firm A's means are 2 and 11; firm B's are 6 and 7), the within estimate is exactly 1.00. The pooled slope was driven by the stable difference between the two firms, not by what happens when a firm changes its R&D.

What fixed effects can and cannot do

• Controls for all time-invariant confounders, measured or not. This is its main strength.
• Cannot estimate the effect of time-invariant variables (sex, country of birth, industry if firms never switch). They are absorbed by α_i and drop out.
• Uses only within-unit variation. If x barely changes over time within units, FE estimates are imprecise.
• Does not control for time-varying confounders. Adding time fixed effects (year dummies), giving a "two-way fixed effects" model, handles shocks common to all units in a given year, but not unit-specific changes.

The random effects model

The random effects (RE) model treats α_i as a random draw from a distribution, usually normal with mean zero and variance σ²_α. Instead of removing the unit effects, it estimates their variance and uses a weighted combination of within-unit and between-unit variation (in practice, a partial or "quasi" demeaning).

Its advantages:

• More efficient (smaller standard errors) than FE, because it uses both kinds of variation.
• Can estimate effects of time-invariant predictors.
• Natural when units are a random sample from a larger population and you want to generalize to it.

Its key assumption: the unit effects α_i are uncorrelated with the predictors. In the R&D example, that means unmeasured management quality is unrelated to R&D spending. If this fails, RE estimates are biased and inconsistent.

A note on terminology: in multilevel (mixed) modeling, "random effects" usually refers to random intercepts and slopes in a hierarchical model. The panel econometrics random effects model is a random-intercept model, so the two are related, and the same assumption about the intercepts being independent of the predictors applies.

Fixed effects vs random effects: which to use

In observational research, where the point of using panel data is usually to control for unobserved heterogeneity, fixed effects is the more conservative choice. Random effects is attractive when its assumption is credible, and in experiments where treatment is randomized and so unrelated to unit traits.

A middle path is the correlated random effects (Mundlak) approach: estimate an RE model that also includes each time-varying predictor's unit mean. The coefficients on the time-varying predictors then match the FE estimates, while time-invariant predictors can still be included.

The Hausman test

The logic

The Hausman test compares the FE and RE estimates of the coefficients on the time-varying predictors:

• Under the null hypothesis (unit effects uncorrelated with predictors), both estimators are consistent, and RE is efficient. The two sets of estimates should be close.
• Under the alternative, only FE is consistent. The estimates will diverge.

So a large, systematic difference between FE and RE estimates is evidence against the RE assumption.

For a single coefficient, the statistic is:

H = (b_FE − b_RE)² / (Var(b_FE) − Var(b_RE))

With several coefficients, it uses the vector of differences and the difference of the covariance matrices. H follows a chi-square distribution with degrees of freedom equal to the number of coefficients compared.

Worked example

In the R&D study (200 firms, 5 years, 1,000 firm-years), suppose the two models give these estimates for log R&D spending:

Then:

• Difference: 0.42 − 0.55 = −0.13, squared = 0.0169
• Variance difference: 0.08² − 0.06² = 0.0064 − 0.0036 = 0.0028
• H = 0.0169 / 0.0028 = 6.04, with 1 degree of freedom

The chi-square critical value at α = .05 with 1 df is 3.84, and the p-value is .014. The test rejects the null: the RE estimate appears to be biased upward, plausibly because firms with better unmeasured characteristics invest more in R&D. You would report the FE estimate, 0.42.

Interpretation and cautions

• Significant Hausman test: evidence that RE is inconsistent; prefer FE (or a correlated random effects model).
• Non-significant test: no evidence against RE. This does not prove the RE assumption is true; the test may lack power, especially with little within-unit variation.
• The standard test assumes RE is fully efficient, which requires homoscedastic, serially uncorrelated errors. With clustered or heteroscedastic errors, use a robust version: the Mundlak approach with cluster-robust standard errors (test whether the unit-mean terms are jointly zero), or an auxiliary-regression version of the test.
• The variance difference can be negative in finite samples, giving a negative or undefined H. Software may warn about this; a robust version avoids the problem.

In Stata: xtreg y x, fe, estimates store fe, xtreg y x, re, estimates store re, hausman fe re. In R with the plm package: fit model = "within" and model = "random", then phtest(fe, re).

Clustered standard errors

Observations from the same unit are rarely independent: a firm's errors in 2022 and 2023 tend to be correlated. Ordinary standard errors ignore this and are usually too small. In panel models, use standard errors clustered by unit, which allow arbitrary correlation within each unit over time and heteroscedasticity across units.

Practical points:

• Cluster at the level at which observations are correlated, often the unit (firm, person) and sometimes a higher level (region, school) if treatment is assigned there.
• Cluster-robust standard errors need a reasonable number of clusters. With few clusters (a common rule of thumb is fewer than about 30 to 50), they can be unreliable, and a wild cluster bootstrap is often recommended.
• Report the number of clusters.

For the cross-sectional analogue of this problem, see the guide to heteroscedasticity and robust standard errors.

How to report fixed and random effects models in APA 7 style

Methods section:

Results section:

Here the clustered standard error (0.10) is a hypothetical value, larger than the conventional 0.08, which is the typical pattern when errors are correlated within firms. Report the test statistic, degrees of freedom, and p-value your software gives; with clustered errors, these depend on the number of clusters. If you suspect within-firm correlation from the start, use a robust version of the Hausman test instead of the conventional one.

Table checklist:

• Which model (FE, RE, two-way FE), and which fixed effects are included (unit, year).
• Coefficients with standard errors (state that they are clustered and at what level).
• Number of observations and number of units (clusters).
• Within R² for FE models (the overall R² is less meaningful there).
• Hausman test statistic, df, and p-value, and whether a robust version was used.

Common mistakes

• Choosing RE because it gives significant results. Its smaller standard errors are only valid if its assumption holds.
• Interpreting FE coefficients as between-unit effects. FE estimates describe what happens when a unit changes, not how units with different levels compare.
• Expecting FE to estimate time-invariant variables. They are absorbed by the unit effects; use RE or correlated random effects if you need them.
• Running the standard Hausman test with clustered errors. Use a robust alternative.
• Forgetting to cluster standard errors by unit.
• Treating a non-significant Hausman test as proof that RE is correct.

FAQ

What is the difference between a fixed effects and a random effects model in one sentence?

Fixed effects removes every unit's stable characteristics and uses only changes within units, while random effects treats unit differences as random noise that must be unrelated to the predictors, and in return uses more of the data.

What does a significant Hausman test mean?

That the FE and RE estimates differ systematically, which suggests the unit effects are correlated with the predictors. In that case, random effects estimates are likely biased, and fixed effects is preferred.

Can I use fixed effects with only two time points?

Yes. With two periods, fixed effects is equivalent to regressing the change in y on the change in x (first differences). It is the same logic as many quasi-experimental designs, including difference-in-differences.

Can I include mediators or interactions in a panel model?

Yes. Interactions between time-varying variables, or between a time-varying and a time-invariant variable, are estimable with FE. Mediation in panel data is more involved; the cross-sectional logic is covered in the regression and mediation analysis guide.

Is a multilevel model the same as a random effects model?

A random-intercept multilevel model for repeated measures is essentially the same model as the panel random effects model, with the same assumption that the intercepts are unrelated to the predictors. Multilevel models can add random slopes and more levels.

From panel data to a paper draft

If you have a complete dataset and want a full draft with the analysis, tables, figures, and write-up, Data2Paper Generate Paper creates one from a CSV or XLSX upload. The first half is free to preview; the full paper costs a one-time $19.90. It is an AI-generated draft, so verify the model specification and every reported number yourself.