Evidence · Meta-analyses

Second pass at: When a network meta-analysis is defensible and when it is not

Evidence › Meta-analyses heterogeneity meta-analysis effect size publication bias risk of bias

Solved Pinned

Solved by Okafor in post #2

Two things before anyone answers the substance. First, the context in the first post is clear and specific. Second, the question is framed so that an answer can actually address it. Both are the norm here and both matter more than they sound.

Jump to the accepted answer →

l.solbergTL2 Moderator3 Oct 2024#1

Posting this under the heading it deserves: Second pass at: When a network meta-analysis is defensible and when it is not Everything below is what sits behind that.

Comparing STEP 8 (JAMA, 2022) with LEADER (N Engl J Med, 2016) and finding the comparison harder than it looks.

Different populations, different durations, different endpoints defined slightly differently, and in one case a different estimand. People compare the headline percentages anyway, including me until recently.

Is there a defensible way to put these side by side, or is the honest answer that there is not and we should stop?

2 likes 22mo

OkaforTL3Regular Solution22 Oct 2024#2

Two things before anyone answers the substance.

First, the context in the first post is clear and specific. Second, the question is framed so that an answer can actually address it. Both are the norm here and both matter more than they sound.

12 likes 21mo

il.dumitruTL2 Moderator4 Nov 2024#3

l.solberg, post #1: Posting this under the heading it deserves: Second pass at: When a network meta-analysis is defensible and when it is not Everything below is what sits behind that. Comparing STEP 8 ( JAMA , 2022) with LEADER ( N Engl J Med , 2016) and finding the comparison harder than it looks. Different populations, different durations, different… Go to post

Picking up post #2: that is the part I would want checked first.

When a meta-analysis is unhelpful: if the included studies are heterogeneous in population, intervention, or outcome, pooling them produces a number that represents nothing in particular. Reading the individual studies is more useful than reading the pooled estimate.

21 likes in reply to #1 21mo

Suggested topics

Topic	Participants	Replies	Views	Activity
Follow-up: Individual participant data versus aggregate data Evidence › Meta-analyses meta-analysis effect size publication bias heterogeneity Individual participant data versus aggregate data — setting out what I have, and where I think it stops being reliable. Comparing LEADER ( N Engl J Med , 2016) with SURPASS-4 ( Lancet , 2021) and finding the…	+101	111	23k	18mo
Heterogeneity as information rather than as a nuisance Evidence › Meta-analyses heterogeneity risk of bias meta-analysis effect size Posting this under the heading it deserves: Heterogeneity as information rather than as a nuisance Everything below is what sits behind that. Session topic: SCALE ( N Engl J Med , 2015). Please read it before…	+88	95	1.2k	6mo
When a network meta-analysis is defensible and when it is not Evidence › Meta-analyses effect size heterogeneity publication bias Solved When a network meta-analysis is defensible and when it is not I have a specific reason for asking rather than idle curiosity, and the context is below. Comparing SUSTAIN 6 ( N Engl J Med , 2016) with LEADER (…		2	27k	4h
Publication bias detection and its low power Evidence › Meta-analyses heterogeneity publication bias effect size On the subject in the title: Publication bias detection and its low power Working notes rather than a conclusion. I have seen SURMOUNT-1 ( N Engl J Med , 2022) cited in support of a claim I do not think it…		2	3.2k	1d
Random versus fixed effects: choosing rather than defaulting Evidence › Meta-analyses effect size publication bias risk of bias heterogeneity Closed Random versus fixed effects: choosing rather than defaulting — setting out what I have, and where I think it stops being reliable. Session topic: FLOW ( N Engl J Med , 2024). Please read it before posting;…	+16	20	22k	13mo

Related topics — sharing the tags heterogeneity, meta-analysis, effect size

Topic	Participants	Replies	Views	Activity
Sample size intuition for a personal experiment — does this still hold? Research Methods › Statistics effect size heterogeneity number needed to treat Solved Sample size intuition for a personal experiment — does this still hold? I have a specific reason for asking rather than idle curiosity, and the context is below. Comparing STEP 1 ( N Engl J Med , 2021) with…	+33	38	26k	23d
Heterogeneity as information rather than as a nuisance Evidence › Meta-analyses heterogeneity risk of bias meta-analysis effect size Posting this under the heading it deserves: Heterogeneity as information rather than as a nuisance Everything below is what sits behind that. Session topic: SCALE ( N Engl J Med , 2015). Please read it before…	+88	95	1.2k	6mo
[2026 update] Does dual agonism explain the effect size, or is it dose? Compounds › Tirzepatide SURMOUNT programme randomised trial SURPASS programme effect size Solved Asking directly, because I could not find a straight answer: Does dual agonism explain the effect size, or is it dose? Session topic: SELECT ( N Engl J Med , 2023). Please read it before posting; the…	+58	63	40k	9mo
Journal club: the CagriSema phase 2 combination paper — a second dataset Evidence › Journal club journal club estimand effect size Solved Journal club: the CagriSema phase 2 combination paper — a second dataset — setting out what I have, and where I think it stops being reliable. I have seen FLOW ( N Engl J Med , 2024) cited in support of a…	+74	79	2.3k	13mo
How this community labels and discusses unrefereed work — the long version Evidence › Preprints needs a source effect size risk of bias Asking directly, because I could not find a straight answer: How this community labels and discusses unrefereed work — the long version I have seen SURMOUNT-1 ( N Engl J Med , 2022) cited in support of a…	+18	23	21k	2mo

Sponsored Sigma-Aldrich — certified reference materialsReference standards, analytical reagents and certified materials for method calibration. If you are validating a method, this is where the standard comes from.Reference standards →