Evidence · Study critique

A structured critique template this community uses

Evidence › Study critique observational data effect size risk of bias

z.okonkwoTL2 Moderator15 Jul 2026#1

A structured critique template this community uses — setting out what I have, and where I think it stops being reliable.

Comparing SURMOUNT-2 (Lancet, 2023) with SURPASS-4 (Lancet, 2021) and finding the comparison harder than it looks.

Different populations, different durations, different endpoints defined slightly differently, and in one case a different estimand. People compare the headline percentages anyway, including me until recently.

Is there a defensible way to put these side by side, or is the honest answer that there is not and we should stop?

7 likes 13d

chromatogramTL4Analytical chemist22 Jul 2026#2

Generalisability: do the inclusion/exclusion criteria narrow the population so much that results do not apply to real people asking about it? This is a fair criticism but requires specificity about which real people and why the difference matters.

0 likes 6d

e.kuuselaTL2 Moderator27 Jul 2026#3

Practical note that does not fit anywhere else. Whatever you conclude from this topic, write down what you did and when. The single most useful thing in your own records is not any individual result; it is that they are dated and consecutive.

32 likes 1d

This topic was referenced in

A claim built entirely on a subgroup analysisEvidence › Study critique · 80 replies

Suggested topics

Topic	Participants	Replies	Views	Activity
A claim built entirely on a subgroup analysis — what changed since Evidence › Study critique disputed effect size observational data risk of bias Closed On the subject in the title: A claim built entirely on a subgroup analysis — what changed since Working notes rather than a conclusion. I have seen PIONEER 6 ( N Engl J Med , 2019) cited in support of a claim…		2	2k	18mo
Critiquing an observational claim about a class effect Evidence › Study critique disputed risk of bias effect size observational data Posting this under the heading it deserves: Critiquing an observational claim about a class effect Everything below is what sits behind that. Session topic: SCALE ( N Engl J Med , 2015). Please read it before…		2	6.9k	14mo
About the Study critique category Evidence › Study critique risk of bias confounding observational data Pinned Wiki Structured criticism of a specific published study. Criticise the method, not the author. This post is a community wiki: any member at trust level 3 or above can edit it, and every edit is recorded with its…		4	114	16mo
Criticising the method without criticising the authors Evidence › Study critique disputed observational data confounding effect size Criticising the method without criticising the authors Writing it up because I had to work it out twice and would rather nobody else did. I have seen STEP 1 ( N Engl J Med , 2021) cited in support of a claim…	+55	61	60k	15mo
Measurement error in a self-reported exposure Evidence › Study critique confounding risk of bias effect size disputed Solved Closed Posting this under the heading it deserves: Measurement error in a self-reported exposure Everything below is what sits behind that. Session topic: SURPASS-2 ( N Engl J Med , 2021). Please read it before…	+117	129	45k	5mo

Related topics — sharing the tags observational data, effect size, risk of bias

Topic	Participants	Replies	Views	Activity
[2026 update] What we do not know about retatrutide, listed explicitly Compounds › Retatrutide TRIUMPH programme preprint effect size randomised trial Wiki What we do not know about retatrutide, listed explicitly — that is the question, and I have not found it answered plainly anywhere I have looked. Comparing SCALE ( N Engl J Med , 2015) with STEP 2 ( Lancet ,…	+79	83	952	11mo
Sample size in a voluntary survey: the selection problem Data & Tools › Datasets observational data data table site feedback Solved Posting this under the heading it deserves: Sample size in a voluntary survey: the selection problem Everything below is what sits behind that. Posting the method first, because I know what the first three…	+1	5	12k	15mo
Absence of evidence and evidence of absence Research Methods › Statistics effect size worked example number needed to treat Absence of evidence and evidence of absence Writing it up because I had to work it out twice and would rather nobody else did. Session topic: SURMOUNT-OSA ( N Engl J Med , 2024). Please read it before…	+16	20	3.9k	5mo
Second pass at: Designing a personal experiment that could change your mind Research Methods › N-of-1 designs worked example effect size confounding data table Solved Second pass at: Designing a personal experiment that could change your mind Writing it up because I had to work it out twice and would rather nobody else did. Session topic: SURPASS-4 ( Lancet , 2021). Please…	+39	44	632	14mo
What pooling buys you and what it destroys Evidence › Meta-analyses publication bias risk of bias meta-analysis heterogeneity Solved The question in the title: What pooling buys you and what it destroys I will give what I have already checked below so nobody repeats it. Comparing SURMOUNT-2 ( Lancet , 2023) with LEADER ( N Engl J Med ,…	+74	83	6.6k	8mo

Sponsored PeptideMeter — peptide analytics and supplier verificationPurity analysis and structured supplier verification, published as dated records rather than scores. Referenced throughout our sourcing documentation.View analytics →