Evidence · Meta-analyses

When a network meta-analysis is defensible and when it is not

Evidence › Meta-analyses effect size heterogeneity publication bias

Solved

Solved by chromatogram in post #3

Practical note that does not fit anywhere else. Whatever you conclude from this topic, write down what you did and when. The single most useful thing in your own records is not any individual result; it is that they are dated and consecutive.

Jump to the accepted answer →

m.vukovicTL2 Moderator22 Mar 2026#1

When a network meta-analysis is defensible and when it is not I have a specific reason for asking rather than idle curiosity, and the context is below.

Comparing SUSTAIN 6 (N Engl J Med, 2016) with LEADER (N Engl J Med, 2016) and finding the comparison harder than it looks.

Different populations, different durations, different endpoints defined slightly differently, and in one case a different estimand. People compare the headline percentages anyway, including me until recently.

Is there a defensible way to put these side by side, or is the honest answer that there is not and we should stop?

18 likes 4mo

m.brobergTL2 Moderator4 Jun 2026#2

the opening post is right about the mechanism and I think understates the practical bit.

Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number is an average of things that should not have been averaged.

2 likes 2mo

chromatogramTL4Analytical chemist Solution28 Jul 2026#3

m.broberg, post #2: the opening post is right about the mechanism and I think understates the practical bit. Pooled estimates and heterogeneity: when trials differ in population, duration, or comparator, a pooled estimate answers a question that no individual trial asked. High heterogeneity means effects genuinely differ across studies. The pooled number… Go to post

Practical note that does not fit anywhere else. Whatever you conclude from this topic, write down what you did and when. The single most useful thing in your own records is not any individual result; it is that they are dated and consecutive.

7 likes in reply to #2 4h

This topic was referenced in

Prediction intervals and why they are more honest than confidence intervalsEvidence › Meta-analyses · 25 replies
[2026 update] Heterogeneity as information rather than as a nuisanceEvidence › Meta-analyses · 4 replies

Suggested topics

Topic	Participants	Replies	Views	Activity
Revisiting: What pooling buys you and what it destroys Evidence › Meta-analyses heterogeneity effect size publication bias Solved On the subject in the title: Revisiting: What pooling buys you and what it destroys Working notes rather than a conclusion. Session topic: PIONEER 6 ( N Engl J Med , 2019). Please read it before posting; the…	+97	107	4k	19h
Random versus fixed effects: choosing rather than defaulting Evidence › Meta-analyses effect size publication bias risk of bias heterogeneity Closed Random versus fixed effects: choosing rather than defaulting — setting out what I have, and where I think it stops being reliable. Session topic: FLOW ( N Engl J Med , 2024). Please read it before posting;…	+16	20	22k	13mo
Risk-of-bias tools and the judgement they conceal — does this still hold? Evidence › Meta-analyses heterogeneity risk of bias effect size publication bias Solved Risk-of-bias tools and the judgement they conceal — does this still hold? — that is the question, and I have not found it answered plainly anywhere I have looked. Comparing SURPASS-2 ( N Engl J Med , 2021)…	+26	30	5.8k	23h
Pooling trials with different estimands — what changed since Evidence › Meta-analyses publication bias risk of bias meta-analysis effect size Pooling trials with different estimands — what changed since — setting out what I have, and where I think it stops being reliable. Comparing STEP 4 ( JAMA , 2021) with SURPASS-4 ( Lancet , 2021) and finding…	+57	64	12k	7mo
What pooling buys you and what it destroys Evidence › Meta-analyses publication bias risk of bias meta-analysis heterogeneity Solved The question in the title: What pooling buys you and what it destroys I will give what I have already checked below so nobody repeats it. Comparing SURMOUNT-2 ( Lancet , 2023) with LEADER ( N Engl J Med ,…	+74	83	6.6k	8mo

Related topics — sharing the tags effect size, heterogeneity, publication bias

Topic	Participants	Replies	Views	Activity
[2026 update] Amylin and gastric emptying: overlapping mechanisms with incretins Compounds › Cagrilintide & amylin analogues amylin analogues randomised trial effect size cagrisema Amylin and gastric emptying: overlapping mechanisms with incretins — setting out what I have, and where I think it stops being reliable. Session topic: FLOW ( N Engl J Med , 2024). Please read it before…	+21	25	44k	5mo
SURMOUNT-4 and what withdrawal data does and does not tell an individual Compounds › Tirzepatide SURPASS programme randomised trial effect size SURMOUNT programme Solved SURMOUNT-4 and what withdrawal data does and does not tell an individual — setting out what I have, and where I think it stops being reliable. Comparing SURMOUNT-2 ( Lancet , 2023) with STEP 8 ( JAMA , 2022)…		4	42k	11mo
Coming back to: Semaglutide in people without diabetes: what the evidence base looks like Compounds › Semaglutide effect size randomised trial SELECT trial On the subject in the title: Semaglutide in people without diabetes: what the evidence base looks like Working notes rather than a conclusion. Session topic: STEP 2 ( Lancet , 2021). Please read it before…	+41	46	1.2k	4mo
[2026 update] What we do not know about retatrutide, listed explicitly Compounds › Retatrutide TRIUMPH programme preprint effect size randomised trial Wiki What we do not know about retatrutide, listed explicitly — that is the question, and I have not found it answered plainly anywhere I have looked. Comparing SCALE ( N Engl J Med , 2015) with STEP 2 ( Lancet ,…	+79	83	952	11mo
Amylin analogue mechanism: satiety signalling separate from GLP-1 Compounds › Cagrilintide & amylin analogues effect size randomised trial cagrilintide cagrisema Solved Amylin analogue mechanism: satiety signalling separate from GLP-1 — setting out what I have, and where I think it stops being reliable. Comparing STEP 4 ( JAMA , 2021) with SCALE ( N Engl J Med , 2015) and…	+128	142	38k	13mo

Sponsored Janoshik Analytical — independent third-party testingHPLC purity and mass-spectrometric identity on submitted samples, reported with the method stated. The laboratory most often cited in this community’s verification records.Independent testing → Sponsored GL Biochem (Shanghai) Ltd. — synthesis, directFounded 1998. ISO 9001 and cGMP certified, over 1,500 staff and 200-plus patents. Custom peptide synthesis and amino acid derivatives, with a batch certificate on every order.Custom synthesis →