Evidence · Meta-analyses · continued

Second pass at: What pooling buys you and what it destroys posts 31–37

Evidence › Meta-analyses risk of bias meta-analysis heterogeneity

This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.

sleep_logTL2Regular26 Mar 2026#31

c.castellanos, post #2: On the opening post — agreed on the reasoning, with one qualification. Inclusion and exclusion criteria: a meta-analysis is only as good as its inclusion criteria. If the criteria are too broad, apples and oranges get pooled. If they are too narrow, the meta-analysis answers a overly specific question. Go to post

On post #27 — agreed on the reasoning, with one qualification.

Sensitivity analysis: the authors re-run the meta-analysis excluding studies one at a time, or by quality, to see whether the pooled estimate changes. Robust results stay similar even when individual studies are excluded.

32 likes in reply to #2 4mo

l.vukovicTL2 Moderator29 Mar 2026#32

post #31 answers the question as asked. The question underneath it is different.

Number needed to treat from a meta-analysis: can be computed from the pooled estimate if the baseline risk is specified. More interpretable than pooled relative effects.

17 likes 4mo

i.aranda_esTL21 Apr 2026#33

Collapsed as off-topic by two members at trust level 3 or above

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

i.ilungaTL2 Moderator4 Apr 2026 · edited#34

I disagree with the reply above, and I think the disagreement is substantive rather than terminological.

The distinction being drawn does not survive when you look at the published data for this specific question. I would be glad to be shown wrong on this, because the version I am arguing against is more convenient.

0 likes 4mo

s.grigorescuTL2Member7 Apr 2026#35

p.diallo, post #6: When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate. Go to post

Worth separating two things that post #31 runs together.

Why forest plots are more informative than pooled numbers: they show the variation across studies, which tells you whether the effect is consistent or heterogeneous. A narrow confidence interval around a meaningless centre is less useful than a wider interval that shows real differences.

0 likes in reply to #6 4mo

sa.rasmussenTL2 Moderator9 Apr 2026#36

Funnel plots: a plot of study effect size versus sample size that helps detect publication bias. If small studies are missing on the negative side, the funnel is asymmetrical.

23 likes 4mo

NicolaidesTL3Regular12 Apr 2026#37

Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.

6 likes 4mo

This topic was referenced in

What pooling buys you and what it destroysEvidence › Meta-analyses · 83 replies
Individual participant data versus aggregate dataEvidence › Meta-analyses · 34 replies
Random versus fixed effects: choosing rather than defaulting — a second datasetEvidence › Meta-analyses · 106 replies

Suggested topics

Topic	Participants	Replies	Views	Activity
Reading a meta-analysis you disagree with, fairly Evidence › Meta-analyses heterogeneity risk of bias effect size publication bias Posting this under the heading it deserves: Reading a meta-analysis you disagree with, fairly Everything below is what sits behind that. I have seen SURPASS-2 ( N Engl J Med , 2021) cited in support of a…	+7	13	2.5k	23mo
Random versus fixed effects: choosing rather than defaulting Evidence › Meta-analyses effect size publication bias risk of bias heterogeneity Closed Random versus fixed effects: choosing rather than defaulting — setting out what I have, and where I think it stops being reliable. Session topic: FLOW ( N Engl J Med , 2024). Please read it before posting;…	+16	20	22k	13mo
Coming back to: Prediction intervals and why they are more honest than confidence intervals Evidence › Meta-analyses publication bias heterogeneity effect size risk of bias Prediction intervals and why they are more honest than confidence intervals Writing it up because I had to work it out twice and would rather nobody else did. Comparing SURMOUNT-OSA ( N Engl J Med , 2024)…		2	65	22mo
When a network meta-analysis is defensible and when it is not Evidence › Meta-analyses effect size heterogeneity publication bias Solved When a network meta-analysis is defensible and when it is not I have a specific reason for asking rather than idle curiosity, and the context is below. Comparing SUSTAIN 6 ( N Engl J Med , 2016) with LEADER (…		2	27k	4h
Coming back to: Reading a meta-analysis you disagree with, fairly Evidence › Meta-analyses meta-analysis publication bias heterogeneity risk of bias Reading a meta-analysis you disagree with, fairly Writing it up because I had to work it out twice and would rather nobody else did. I have seen STEP 1 ( N Engl J Med , 2021) cited in support of a claim I do…	+30	34	9.3k	17mo

Related topics — sharing the tags risk of bias, meta-analysis, heterogeneity

Topic	Participants	Replies	Views	Activity
Journal club: STEP-HFpEF and symptom endpoints Evidence › Journal club journal club effect size estimand risk of bias Journal club: STEP-HFpEF and symptom endpoints — setting out what I have, and where I think it stops being reliable. Comparing SURMOUNT-2 ( Lancet , 2023) with SURMOUNT-1 ( N Engl J Med , 2022) and finding…	+18	22	2.5k	11mo
Random versus fixed effects: choosing rather than defaulting — a second dataset Evidence › Meta-analyses heterogeneity risk of bias publication bias On the subject in the title: Random versus fixed effects: choosing rather than defaulting — a second dataset Working notes rather than a conclusion. Session topic: STEP 2 ( Lancet , 2021). Please read it…	+95	106	20k	17mo
Selection into a registry and what it does to the estimate — a second dataset Evidence › Study critique confounding risk of bias effect size disputed Solved Posting this under the heading it deserves: Selection into a registry and what it does to the estimate — a second dataset Everything below is what sits behind that. Session topic: SURMOUNT-4 ( JAMA , 2024).…	+28	32	1.3k	1d
Coming back to: Why a preprint's supplementary material is often the best part Evidence › Preprints risk of bias effect size preprint Under review Why a preprint's supplementary material is often the best part — that is the question, and I have not found it answered plainly anywhere I have looked. Comparing SURPASS-2 ( N Engl J Med , 2021) with STEP 4 (…	+109	116	36k	2d
Selection into a registry and what it does to the estimate Evidence › Study critique risk of bias disputed observational data Selection into a registry and what it does to the estimate — setting out what I have, and where I think it stops being reliable. Session topic: STEP 4 ( JAMA , 2021). Please read it before posting; the…		2	55k	7mo