The Peptide CommonsEst. May 2024
Independent. We sell nothing and are affiliated with no manufacturer or pharmacy. Every moderation action is logged in public
Evidence · Meta-analyses

What pooling buys you and what it destroys

Solved
Solved by n.abernathy in post #5
Coming back to post #3, because the follow-up matters more than the original answer. Publication bias: what did not get published? Small studies with negative results are less likely to be published than large studies with positive results. A forest plot with only large studies on the positive end is a red flag for…

Jump to the accepted answer →

MD
m.dalgaardTL3Regular3 Aug 2025#1

The question in the title: What pooling buys you and what it destroys I will give what I have already checked below so nobody repeats it.

Comparing SURMOUNT-2 (Lancet, 2023) with LEADER (N Engl J Med, 2016) and finding the comparison harder than it looks.

Different populations, different durations, different endpoints defined slightly differently, and in one case a different estimand. People compare the headline percentages anyway, including me until recently.

Is there a defensible way to put these side by side, or is the honest answer that there is not and we should stop?

24 likes 12mo
EH
e.halonenTL2 Moderator7 Aug 2025#2

This follows the opening post rather than contradicting it.

Inclusion and exclusion criteria: a meta-analysis is only as good as its inclusion criteria. If the criteria are too broad, apples and oranges get pooled. If they are too narrow, the meta-analysis answers a overly specific question.

4 likes 12mo
DH
dietitian_hollisTL3Dietitian10 Aug 2025#3

Worth separating two things that the opening post runs together.

Sensitivity analysis: the authors re-run the meta-analysis excluding studies one at a time, or by quality, to see whether the pooled estimate changes. Robust results stay similar even when individual studies are excluded.

0 likes 12mo
PM
p.mwangiTL2 Moderator13 Aug 2025 · edited#4

Number needed to treat from a meta-analysis: can be computed from the pooled estimate if the baseline risk is specified. More interpretable than pooled relative effects.

0 likes 11mo
NA
n.abernathyTL3Analytical chemist Solution15 Aug 2025#5
m.dalgaard, post #1: The question in the title: What pooling buys you and what it destroys I will give what I have already checked below so nobody repeats it. Comparing SURMOUNT-2 ( Lancet , 2023) with LEADER ( N Engl J Med , 2016) and finding the comparison harder than it looks. Different populations, different durations, different endpoints defined… Go to post

Coming back to post #3, because the follow-up matters more than the original answer.

Publication bias: what did not get published? Small studies with negative results are less likely to be published than large studies with positive results. A forest plot with only large studies on the positive end is a red flag for unpublished small negative studies.

18 likes in reply to #1 11mo
NK
n.kuuselaTL2 Moderator17 Aug 2025#6

Picking up post #3: that is the part I would want checked first.

I disagree with the reply above, and I think the disagreement is substantive rather than terminological.

The distinction being drawn does not survive when you look at the published data for this specific question. I would be glad to be shown wrong on this, because the version I am arguing against is more convenient.

7 likes 11mo
PW
PharmNotes_WhitfieldTL4Pharmacist19 Aug 2025#7

Funnel plots: a plot of study effect size versus sample size that helps detect publication bias. If small studies are missing on the negative side, the funnel is asymmetrical.

1 like 11mo
SC
s.cabreraTL2 Moderator21 Aug 2025#8

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

0 likes 11mo
DO
dr_okonkwoTL4 Moderator23 Aug 2025#9
Staff post. Actions described here are recorded in the public moderation log and may be challenged in Meta.

I read post #7 twice before replying, because I had assumed the opposite.

Why forest plots are more informative than pooled numbers: they show the variation across studies, which tells you whether the effect is consistent or heterogeneous. A narrow confidence interval around a meaningless centre is less useful than a wider interval that shows real differences.

4 likes 11mo
CG
c.grimaldiTL2 Moderator25 Aug 2025#10
dietitian_hollis, post #3: Worth separating two things that the opening post runs together. Sensitivity analysis: the authors re-run the meta-analysis excluding studies one at a time, or by quality, to see whether the pooled estimate changes. Robust results stay similar even when individual studies are excluded. Go to post

Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.

0 likes in reply to #3 11mo
RL
r.lundgrenTL2 Moderator27 Aug 2025#11

Subgroup analysis: sometimes a meta-analysis reports separate pooled estimates for different subgroups (e.g., by baseline body mass index or by trial duration). Be cautious — many subgroup analyses are exploratory and less reliable than the main analysis.

21 likes 11mo
DM
d.moreauTL2Regular29 Aug 2025#12

I read post #10 twice before replying, because I had assumed the opposite.

When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.

0 likes 11mo
SA
s.antonsenTL2 Moderator31 Aug 2025#13
c.grimaldi, post #10: Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high. Go to post

post #12 is right about the mechanism and I think understates the practical bit.

Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.

1 like in reply to #10 11mo
QZ
q.zhao_qaTL3Quality assurance2 Sep 2025#14

Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high.

5 likes 11mo
BD
b.dumitruTL2 Moderator3 Sep 2025#15

Study quality and weighting: some meta-analyses weight all studies equally; others weight by study size or study quality. The choice affects the result and should be stated and justified.

29 likes 11mo
NT
nl_translatorTL2Translator · NL5 Sep 2025#16
n.kuusela, post #6: Picking up post #3: that is the part I would want checked first. I disagree with the reply above, and I think the disagreement is substantive rather than terminological. The distinction being drawn does not survive when you look at the published data for this specific question. I would be glad to be shown wrong on this, because the… Go to post

Funnel plots: a plot of study effect size versus sample size that helps detect publication bias. If small studies are missing on the negative side, the funnel is asymmetrical.

0 likes in reply to #6 11mo
ZC
z.cardosoTL2 Moderator7 Sep 2025#17
d.moreau, post #12: I read post #10 twice before replying, because I had assumed the opposite. When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled… Go to post

post #16 answers the question as asked. The question underneath it is different.

Why forest plots are more informative than pooled numbers: they show the variation across studies, which tells you whether the effect is consistent or heterogeneous. A narrow confidence interval around a meaningless centre is less useful than a wider interval that shows real differences.

2 likes in reply to #12 11mo
EN
electrolyte_notesTL2Regular8 Sep 2025#18

On post #14 — agreed on the reasoning, with one qualification.

Practical note that does not fit anywhere else. Whatever you conclude from this topic, write down what you did and when. The single most useful thing in your own records is not any individual result; it is that they are dated and consecutive.

9 likes 11mo
CO
c.ostergaardTL2 Moderator10 Sep 2025 · edited#19

This follows post #16 rather than contradicting it.

Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.

10 likes 11mo
MS
m.strand_rphTL3Pharmacist12 Sep 2025#20
q.zhao_qa, post #14: Fixed-effects versus random-effects models: fixed-effects assumes all studies are estimating the same thing and variation is sampling error. Random-effects assumes studies are estimating effects from different distributions and allows between-study variance. Choice matters if heterogeneity is high. Go to post

When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.

22 likes in reply to #14 10mo
AZ
a.zamoraTL213 Sep 2025#21
CS
c.silvaTL2 Moderator15 Sep 2025#22

Publication bias: what did not get published? Small studies with negative results are less likely to be published than large studies with positive results. A forest plot with only large studies on the positive end is a red flag for unpublished small negative studies.

15 likes 10mo
RG
r.girardTL2 Moderator16 Sep 2025 · edited#23

Worth separating two things that post #19 runs together.

Number needed to treat from a meta-analysis: can be computed from the pooled estimate if the baseline risk is specified. More interpretable than pooled relative effects.

3 likes 10mo
NN
n.norgaardTL2 Moderator18 Sep 2025#24
z.cardoso, post #17: post #16 answers the question as asked. The question underneath it is different. Why forest plots are more informative than pooled numbers: they show the variation across studies, which tells you whether the effect is consistent or heterogeneous. A narrow confidence interval around a meaningless centre is less useful than a wider… Go to post

post #23 is right about the mechanism and I think understates the practical bit.

Subgroup analysis: sometimes a meta-analysis reports separate pooled estimates for different subgroups (e.g., by baseline body mass index or by trial duration). Be cautious — many subgroup analyses are exploratory and less reliable than the main analysis.

0 likes in reply to #17 10mo
JV
j.vandermolenTL3Regular20 Sep 2025#25

Coming back to post #23, because the follow-up matters more than the original answer.

Inclusion and exclusion criteria: a meta-analysis is only as good as its inclusion criteria. If the criteria are too broad, apples and oranges get pooled. If they are too narrow, the meta-analysis answers a overly specific question.

23 likes 10mo
AL
a.lindqvistTL2 Moderator21 Sep 2025#26

Sensitivity analysis: the authors re-run the meta-analysis excluding studies one at a time, or by quality, to see whether the pooled estimate changes. Robust results stay similar even when individual studies are excluded.

10 likes 10mo
GV
g.valckenaereTL3Regular23 Sep 2025 · edited#27
c.silva, post #22: Publication bias: what did not get published? Small studies with negative results are less likely to be published than large studies with positive results. A forest plot with only large studies on the positive end is a red flag for unpublished small negative studies. Go to post

Publication bias: what did not get published? Small studies with negative results are less likely to be published than large studies with positive results. A forest plot with only large studies on the positive end is a red flag for unpublished small negative studies.

1 like in reply to #22 10mo
SR
s.roosTL2 Moderator24 Sep 2025#28
z.cardoso, post #17: post #16 answers the question as asked. The question underneath it is different. Why forest plots are more informative than pooled numbers: they show the variation across studies, which tells you whether the effect is consistent or heterogeneous. A narrow confidence interval around a meaningless centre is less useful than a wider… Go to post

post #27 answers the question as asked. The question underneath it is different.

I disagree with the reply above, and I think the disagreement is substantive rather than terminological.

The distinction being drawn does not survive when you look at the published data for this specific question. I would be glad to be shown wrong on this, because the version I am arguing against is more convenient.

0 likes in reply to #17 10mo
SO
s.okaforTL2 Moderator26 Sep 2025#29
n.abernathy, post #5: Coming back to post #3, because the follow-up matters more than the original answer. Publication bias: what did not get published? Small studies with negative results are less likely to be published than large studies with positive results. A forest plot with only large studies on the positive end is a red flag for unpublished small… Go to post

Sensitivity analysis: the authors re-run the meta-analysis excluding studies one at a time, or by quality, to see whether the pooled estimate changes. Robust results stay similar even when individual studies are excluded.

16 likes in reply to #5 10mo
RA
r.aldana_pharmdTL4Pharmacist27 Sep 2025#30

This follows post #27 rather than contradicting it.

Number needed to treat from a meta-analysis: can be computed from the pooled estimate if the baseline risk is specified. More interpretable than pooled relative effects.

6 likes 10mo