Meta-Analysis: Pooling Studies (Fixed vs Random Effects)

TL;DR — A meta-analysis combines many studies into one estimate so you borrow strength from all of them instead of trusting a single trial. The engine is inverse-variance weighting: each study's weight is w = 1/SE², so precise (large, tight-CI) studies count more and noisy ones count less. A fixed-effect model assumes every study estimates the same true effect and only sampling noise separates them. A random-effects model (DerSimonian-Laird) assumes true effects vary across studies and adds a between-study variance τ² to every weight (w* = 1/(SE² + τ²)), which widens the pooled CI and pulls the weights toward equal. For ratio measures (OR, RR, HR) you must pool on the log scale — combine ln(θ), then exponentiate the pooled estimate and its CI. Report the pooled effect with its CI, and never let one giant study silently dominate.

---

1. Simple explanation

One trial is a single witness. It might be small, unlucky, or run at an unusual hospital. A meta-analysis interviews every witness at once and produces a combined verdict that is more precise than any single account — provided you weigh each witness by how reliable they are.

Analogy — weighing several bathroom scales. You want your true weight and you have four scales. A cheap wobbly scale reads ±3 kg; an expensive lab scale reads ±0.2 kg. You would be foolish to average the four readings equally — that lets the wobbly scale drag the answer around. Instead you weight each reading by its precision: the lab scale, being far more certain, gets far more say. That is exactly inverse-variance weighting — weight = 1 over the reading's variance. A precise study (narrow CI, small SE) gets a big weight; a noisy study gets a small one.

Now the deeper question: are all four scales measuring the same thing? If they are all trying to read your weight and differ only by random jitter, that is the fixed-effect world — one truth, noise around it. But suppose one scale was actually in a different room measuring a slightly different object. Then the scales disagree for a real reason, not just noise, and you should (a) admit there is a spread of true values and (b) be less confident about the single combined number. That is the random-effects world: it adds an extra variance term, τ², for the genuine spread, which makes each weight smaller and the final interval wider and more honest.

The three questions this article answers: How are study weights set? What is the difference between fixed and random effects? And what does the pooled interval actually mean?

---

2. Diagram

        FOUR STUDIES  →  ONE POOLED ESTIMATE (inverse-variance weighting)

   study i:  effect θ_i (log scale for ratios)   with std error SE_i
             weight  w_i = 1 / SE_i²      ← precise study ⇒ big weight

   ┌────────────────────────────────────────────────────────────────┐
   │  FIXED-EFFECT (one true effect, only sampling noise)             │
   │      θ_pooled = Σ w_i θ_i / Σ w_i                                │
   │      SE_pooled = sqrt( 1 / Σ w_i )                               │
   └────────────────────────────────────────────────────────────────┘
                    │  but do the studies really agree?
                    ▼
         Cochran's Q = Σ w_i (θ_i − θ_pooled)²          (heterogeneity signal)
                    │
        τ² (DL) = max(0, (Q − df) / (Σw − Σw²/Σw))      between-study variance
                    │
   ┌────────────────────────────────────────────────────────────────┐
   │  RANDOM-EFFECTS (true effects VARY; add τ² to every weight)     │
   │      w_i* = 1 / (SE_i² + τ²)      ← weights move toward EQUAL    │
   │      θ_pooled* = Σ w_i* θ_i / Σ w_i*                             │
   │      SE_pooled* = sqrt( 1 / Σ w_i* )   ← WIDER interval          │
   └────────────────────────────────────────────────────────────────┘

   for RATIOS (OR/RR/HR):  pool ln(θ),  then report exp(θ_pooled) and exp(CI)

     log-OR axis:  −1.0     −0.5      0.0
                    ├─────────┼─────────┤
       study 1  ────■────                 w
       study 2      ───■──                 (box size ∝ weight)
       study 3  ─■──                       ▲ no-effect line (lnOR = 0)
       study 4         ──■─
       POOLED           ◆  (diamond = pooled estimate + CI)

---

3. How it works

3.1 Why pool at all

A single study answers "what happened here." A meta-analysis answers "what happens in general," with a tighter interval than any one study could give. Pooling four studies of 100 patients is not identical to one study of 400 (design and populations differ), but done carefully it increases precision, exposes disagreement, and averages out one-off flukes. It is the top of the evidence hierarchy for exactly this reason.

3.2 Inverse-variance weighting — the universal recipe

Every study contributes an effect estimate θ_i and its standard error SE_i. The weight is:

w_i = 1 / SE_i²

Smaller SE (bigger, more precise study) → bigger weight. The pooled estimate is the weighted average Σ w_i θ_i / Σ w_i. This is not arbitrary: inverse-variance weights are the choice that minimizes the variance of the pooled estimate — provably the most precise linear combination. It is the same math as the weighted-average idea, applied to whole studies.

3.3 Fixed-effect vs random-effects — the core decision

The practical rule: if there is any real clinical or methodological diversity among the studies — different countries, doses, definitions — the effects almost certainly differ, so random-effects is the honest default. Fixed-effect is appropriate when the studies are near-replicas or when you specifically want a within-those-exact-studies summary.

3.4 Estimating τ² with DerSimonian-Laird

Random-effects needs a number for the between-study variance. The DerSimonian-Laird (DL) method reads it off Cochran's Q, a weighted sum of squared deviations from the fixed pooled estimate:

Q  = Σ w_i (θ_i − θ_pooled)²          (using fixed-effect weights)
df = k − 1                            (k = number of studies)
τ² = max(0, (Q − df) / (Σw − Σw²/Σw))

The logic: if there were no real heterogeneity, Q would on average equal its degrees of freedom df. Any excess of Q over df is attributed to between-study variance. The max(0, …) floor means τ² is never negative (you cannot have negative variance) — if Q < df, τ² is set to 0 and random-effects collapses back to fixed-effect. Q and its companion I² are the subject of the next article; here we only need Q to get τ².

3.5 The "combine on the log scale" rule for ratios

RR, OR, and HR are ratios, whose sampling distributions are skewed — you cannot average them directly (the average of 0.5 and 2.0 is not the "no-effect" 1.0 it should be). So for ratios you:

1. Convert each study's estimate to θ_i = ln(ratio_i), with SE_i already on the log scale.
2. Pool θ_i with inverse-variance weighting → θ_pooled.
3. Report exp(θ_pooled) and exp(θ_pooled ± 1.96·SE_pooled).

Differences (RD, mean differences) are already symmetric and are pooled on their natural scale — no log.

3.6 What the pooled CI means

The pooled 95% CI is the range of pooled effects compatible with the combined data at the 5% level. If it excludes the no-effect value (ratio = 1, difference = 0), the pooled effect is significant. Crucially, under random-effects the CI is a statement about the average effect across the distribution of study settings, not a single universal truth — and it is wider precisely because it folds in τ². A common mistake is to read the random-effects CI as a prediction interval for a new study; it is not (a prediction interval is a separate, even wider band).

---

4. The math

w_i        = 1 / SE_i²
θ_pooled   = Σ w_i θ_i / Σ w_i           (fixed)
SE_pooled  = sqrt( 1 / Σ w_i )
95% CI     = θ_pooled ± 1.96 · SE_pooled      (exponentiate for ratios)

Q          = Σ w_i (θ_i − θ_pooled)²
τ²         = max(0, (Q − df) / (Σw − Σw²/Σw)),   df = k − 1
w_i*       = 1 / (SE_i² + τ²)            (random-effects weights)
θ_pooled*  = Σ w_i* θ_i / Σ w_i*
SE_pooled* = sqrt( 1 / Σ w_i* )

Worked numeric example — pooling four odds-ratio studies. Each study reports an OR; convert to ln(OR) and its log-scale SE (values rounded).

Fixed-effect pool:

Σw       = 25 + 16 + 11.11 + 44.44 = 96.56
Σ w·θ    = 25(−0.51) + 16(−0.36) + 11.11(−0.92) + 44.44(−0.11)
         = −12.75 − 5.76 − 10.22 − 4.89 = −33.62
θ_pooled = −33.62 / 96.56 = −0.348
OR_fixed = exp(−0.348) = 0.706

SE_pooled = sqrt(1/96.56) = 0.1018
95% CI    = exp(−0.348 ± 1.96·0.1018) = exp(−0.348 ± 0.1995)
          = (exp(−0.548), exp(−0.149)) = (0.578, 0.862)
FIXED:  OR = 0.71  (95% CI 0.58 to 0.86)

Heterogeneity → τ²:

Q  = 25(−0.51+0.348)² + 16(−0.36+0.348)² + 11.11(−0.92+0.348)² + 44.44(−0.11+0.348)²
   = 25(0.0262) + 16(0.00014) + 11.11(0.3270) + 44.44(0.0567)
   = 0.654 + 0.002 + 3.633 + 2.522 = 6.81
df = 3

Σw² = 25² + 16² + 11.11² + 44.44² = 625 + 256 + 123.5 + 1975.3 = 2979.7
τ²  = max(0, (6.81 − 3) / (96.56 − 2979.7/96.56))
    = (3.81) / (96.56 − 30.86) = 3.81 / 65.70 = 0.0580

Random-effects pool (add τ² = 0.058 to each SE²):

w_i* = 1/(SE_i² + 0.058):
   1: 1/(0.040+0.058)=10.20   2: 1/(0.0625+0.058)=8.30
   3: 1/(0.090+0.058)=6.76    4: 1/(0.0225+0.058)=12.42
Σw*  = 37.68

Σ w*·θ = 10.20(−0.51) + 8.30(−0.36) + 6.76(−0.92) + 12.42(−0.11) = −15.77
θ_pooled* = −15.77 / 37.68 = −0.419
OR_random = exp(−0.419) = 0.658

SE_pooled* = sqrt(1/37.68) = 0.1629
95% CI     = exp(−0.419 ± 1.96·0.1629) = (exp(−0.738), exp(−0.099))
           = (0.478, 0.906)
RANDOM:  OR = 0.66  (95% CI 0.48 to 0.91)

Read it. Fixed gives OR 0.71 (0.58–0.86); random gives OR 0.66 (0.48–0.91). Two things happened when we switched to random-effects: (1) the CI widened (0.58–0.86 → 0.48–0.91) because τ² added real uncertainty, and (2) the point estimate shifted toward the smaller studies. Notice study 4 (the biggest, w = 44.4) held 46% of the fixed weight but only 33% of the random weight — adding τ² pulled the weights toward equal, giving the smaller, more extreme study 3 a bigger voice. Both CIs still exclude 1, so the pooled protective effect is significant either way.

---

5. Real code

Pure numpy. Give it arrays of effects (log scale for ratios) and their SEs; it returns fixed- and random-effects pools with CIs, plus Q, I², and τ².

"""Inverse-variance meta-analysis: fixed-effect and DerSimonian-Laird random-effects.
Pass effects on the LOG scale for ratio measures (OR/RR/HR); exponentiate the output.
Returns pooled effect, 95% CI, Cochran's Q, I%, and between-study variance tau^2."""
import numpy as np

Z = 1.959964  # 95% two-sided normal multiplier

def meta_analyze(effects, ses):
    """effects, ses: 1-D arrays. Effects on the analysis scale (log scale for ratios)."""
    theta = np.asarray(effects, float)
    se = np.asarray(ses, float)
    w = 1.0 / se**2                       # inverse-variance weights
    k = len(theta)

    # ---- fixed-effect pool ----
    theta_fixed = np.sum(w * theta) / np.sum(w)
    se_fixed = np.sqrt(1.0 / np.sum(w))

    # ---- heterogeneity: Cochran's Q, I2, DL tau^2 ----
    Q = np.sum(w * (theta - theta_fixed)**2)
    df = k - 1
    I2 = max(0.0, (Q - df) / Q) * 100 if Q > 0 else 0.0
    denom = np.sum(w) - np.sum(w**2) / np.sum(w)
    tau2 = max(0.0, (Q - df) / denom) if denom > 0 else 0.0

    # ---- random-effects pool (weights include tau^2) ----
    w_star = 1.0 / (se**2 + tau2)
    theta_random = np.sum(w_star * theta) / np.sum(w_star)
    se_random = np.sqrt(1.0 / np.sum(w_star))

    def ci(theta_hat, se_hat):
        return theta_hat - Z*se_hat, theta_hat + Z*se_hat

    return {
        "fixed":  (theta_fixed,  *ci(theta_fixed,  se_fixed),  w / w.sum()),
        "random": (theta_random, *ci(theta_random, se_random), w_star / w_star.sum()),
        "Q": Q, "df": df, "I2": I2, "tau2": tau2,
    }

if __name__ == "__main__":
    # four odds-ratio studies, converted to the log scale
    ORs = np.array([0.60, 0.70, 0.40, 0.90])
    theta = np.log(ORs)
    se = np.array([0.20, 0.25, 0.30, 0.15])

    r = meta_analyze(theta, se)
    for model in ("fixed", "random"):
        est, lo, hi, weights = r[model]
        print(f"{model:6s}: OR = {np.exp(est):.3f}  "
              f"(95% CI {np.exp(lo):.3f} to {np.exp(hi):.3f})  "
              f"weights % = {np.round(weights*100, 1)}")
    print(f"Q = {r['Q']:.2f}  df = {r['df']}  I2 = {r['I2']:.1f}%  tau2 = {r['tau2']:.4f}")

Expected output:

fixed : OR = 0.706  (95% CI 0.578 to 0.862)  weights % = [25.9 16.6 11.5 46. ]
random: OR = 0.658  (95% CI 0.478 to 0.906)  weights % = [27.1 22.  17.9 33. ]
Q = 6.81  df = 3  I2 = 56.0%  tau2 = 0.0580

The printed weight rows make the fixed-to-random shift concrete: study 4 drops from 46% to 33% of the total weight while the small studies rise — the mechanical effect of adding τ² to every denominator.

---

6. Real-world example

A systematic review of a blood-pressure drug across four countries.

• Setup. Four RCTs test the same drug class for preventing stroke, but in different countries, with slightly different doses and patient ages. Reported odds ratios: 0.60, 0.70, 0.40, 0.90, with log-scale SEs 0.20, 0.25, 0.30, 0.15. Study 4 is the largest and most precise; study 3 is small but reports the strongest effect.
• First pass — fixed-effect. Pooled OR = 0.71 (0.58–0.86). Precise, significant — but it assumes all four trials estimate the identical true effect, which is implausible given four countries and doses.
• Check the assumption. Cochran's Q = 6.81 on df = 3, I² = 56% — moderate heterogeneity. The trials genuinely disagree beyond chance, so the fixed-effect assumption is unsafe. (The full read on Q/I² is the next article.)
• Second pass — random-effects. τ² = 0.058, pooled OR = 0.66 (0.48–0.91). The point estimate moves toward the smaller trials and the CI widens by roughly 50% — the honest price of admitting the true effect varies across settings.
• Interpretation. The headline is the random-effects OR 0.66 (0.48–0.91): on average across these settings the drug cuts stroke odds by about a third, and the effect is significant (CI excludes 1). Report it as an average effect, not a universal constant, and flag that a new population could land anywhere in a still-wider prediction interval.
• Why not just trust study 4? Study 4 alone (OR 0.90) would suggest almost no effect. Pooling shows that its precision is real but its result is an outlier relative to the other three; the combined evidence is stronger and more protective than the single largest trial.
• Decision. The review reports the random-effects estimate as primary, the fixed-effect as a sensitivity check, and hands the heterogeneity investigation (subgroups by dose/age) to the next stage.

---

7. Interview questions companies actually ask

Q [Cochrane / evidence synthesis] "Fixed-effect vs random-effects — how do you choose?"
  A Fixed-effect assumes one true effect and only sampling noise; random-effects assumes true
    effects vary across studies and adds between-study variance tau^2 to every weight. If the
    studies differ in population, dose, or setting — the usual case — the effects almost certainly
    differ, so random-effects is the honest default. Use fixed-effect only when the studies are
    near-replicas or you want a summary of exactly those studies. Random-effects gives a wider,
    more conservative CI.

Q [a health-insurance analytics team] "How are study weights set in a meta-analysis, and why?"
  A By inverse variance: w = 1/SE^2. Precise studies (large, tight CI) get big weights; noisy
    studies get small ones. This isn't arbitrary — inverse-variance weights minimize the variance
    of the pooled estimate, so they give the most precise linear combination. Random-effects
    modifies it to w* = 1/(SE^2 + tau^2), which shrinks the weights toward equal and gives small
    studies relatively more say.

Q [Johnson & Johnson / medical affairs] "What does the pooled confidence interval actually mean?"
  A It's the range of pooled effects compatible with the combined data at the 5% level; if it
    excludes the no-effect value (ratio 1, difference 0), the pooled effect is significant. Under
    random-effects it describes the AVERAGE effect across the distribution of settings and is
    wider because it includes tau^2. It is NOT a prediction interval for a new study — that band
    is wider still.

Q [a CRO biostatistician] "Why do you pool odds ratios on the log scale?"
  A Ratios are skewed and multiplicative — averaging them directly is invalid (the mean of 0.5 and
    2.0 isn't the no-effect 1.0). Taking logs makes the effects additive and roughly normal, so
    inverse-variance pooling behaves. You combine ln(OR) with log-scale SEs, then exponentiate the
    pooled estimate and its CI. Risk differences and mean differences are already symmetric and
    are pooled on the natural scale.

Q [Novartis] "Your largest trial holds 46% of the weight and disagrees with the rest. What happens
   when you switch to random-effects?"
  A Adding tau^2 to every denominator shrinks all weights toward equal, so the dominant trial
    loses relative influence (e.g. 46% down to 33%) and the smaller trials gain. The pooled point
    estimate moves toward the smaller studies and the CI widens. This is desirable when there's
    real heterogeneity — you don't want one big trial silently deciding the answer.

Q [a public-health agency] "How does DerSimonian-Laird estimate tau^2?"
  A From Cochran's Q. Under no heterogeneity, Q averages its degrees of freedom df = k−1. DL
    attributes the excess to between-study variance: tau^2 = max(0, (Q − df)/(Σw − Σw²/Σw)). The
    max(0,·) floor stops it going negative; if Q < df, tau^2 = 0 and random-effects reduces to
    fixed-effect. DL is a moment estimator — simple and common, though REML or Paule-Mandel are
    often preferred with few studies.

Q [Amgen] "Is pooling four 100-patient trials the same as one 400-patient trial?"
  A No. Meta-analysis increases precision and averages out flukes, but the studies differ in
    population, protocol, and quality, and random-effects deliberately keeps their true effects
    distinct via tau^2. You also can't inspect patient-level interactions the way one big trial
    could. It's stronger than any single trial but it's a synthesis of heterogeneous evidence, not
    a mega-trial.

Q [a regulatory reviewer] "When would fixed and random effects give nearly identical answers?"
  A When heterogeneity is low — Q near df, I^2 near 0, so tau^2 ~ 0 and w* ~ w. Then the two
    models coincide. They also converge when one study is so much more precise than the others
    that it dominates regardless. They diverge most when heterogeneity is high AND weights are
    uneven, which is exactly when the choice matters and you should prefer random-effects.

---

8. When to use / tradeoffs

   USE META-ANALYSIS WHEN:
     ✓ several comparable studies estimate the same question — pool for precision
     ✓ you want an average effect + honest uncertainty across settings
     ✓ you need a single summary to feed a guideline or decision
   FIXED vs RANDOM:
     ✓ fixed-effect  — near-replica studies, or a summary of exactly these studies
     ✓ random-effects — any real clinical/methodological diversity (the usual default)
   HONEST LIMITS:
     ✗ garbage in, garbage out — pooling biased studies gives a precise wrong answer
     ✗ publication bias: missing negative studies inflate the pooled effect (check funnel plots)
     ✗ DL tau^2 is unstable with very few studies (k < 5); consider REML / Paule-Mandel
     ✗ a dominant huge study can drive the fixed-effect pool — inspect the weights
     ✗ the pooled CI is NOT a prediction interval for a new study
   THE RULE:
     always report the pooled effect WITH its CI, state fixed vs random, and show the weights.

Pooling multiplies both signal and error: it sharpens a good body of evidence and launders a biased one into a confident mistake. It assumes the studies are combinable, that none is fatally biased, and that the missing (unpublished) studies would not overturn the picture. The moment the studies disagree beyond chance, the fixed-effect summary becomes misleading — which is why the next article is entirely about measuring and reading that disagreement.

---

9. Summary + related articles

• A meta-analysis pools studies to gain precision, expose disagreement, and average out flukes.
• Inverse-variance weighting (w = 1/SE²) gives precise studies more say and is the minimum-variance combination.
• Fixed-effect assumes one true effect; random-effects (DerSimonian-Laird) adds between-study variance τ² (w* = 1/(SE² + τ²)), widening the CI and pulling weights toward equal.
• τ² is estimated from Cochran's Q: τ² = max(0, (Q − df)/(Σw − Σw²/Σw)).
• Pool ratios on the log scale (ln θ), then exponentiate the pooled estimate and CI; differences pool on the natural scale.
• The pooled CI describes the (average) effect; under random-effects it is wider and is not a prediction interval for a new study.
• Report the pooled effect with its CI, state fixed vs random, and inspect the weights.

Related: 2×2 Tables & Effect Measures (OR, RR, RD, NNT, HR) · Heterogeneity & Forest Plots · Probability & Statistics Foundations

Resources

• DerSimonian & Laird, "Meta-analysis in clinical trials" — https://doi.org/10.1016/0197-2456(86)90046-2
• Cochrane Handbook, Ch. 10 "Analysing data and undertaking meta-analyses" — https://training.cochrane.org/handbook/current/chapter-10
• Borenstein, Hedges, Higgins & Rothstein, Introduction to Meta-Analysis
• Veroniki et al., "Methods to estimate the between-study variance and its uncertainty" — https://doi.org/10.1002/jrsm.1164
• IntHout, Ioannidis & Borm, "The Hartung-Knapp-Sidik-Jonkman method" — https://doi.org/10.1186/1471-2288-14-25