Research Methods · Statistics

Second pass at: Multiplicity when you track fifteen variables

Research Methods › Statistics number needed to treat confounding effect size worked example heterogeneity

k.fonsecaTL2 Moderator20 Oct 2024#1

Posting this under the heading it deserves: Second pass at: Multiplicity when you track fifteen variables Everything below is what sits behind that.

I have seen SCALE (N Engl J Med, 2015) cited in support of a claim I do not think it supports, twice this month, so I would like to work through what it actually shows.

My reading is that the trial is sound for its own question and is being stretched to answer a different one. I might be wrong about that, which is why this is a topic rather than a correction.

What I would like from this discussion: someone who disagrees with me to say why, with the section of the paper they are relying on.

60 likes 21mo

f.fontaineTL2 Moderator20 Dec 2024#2

Effect sizes: the magnitude of a difference, not just whether it is statistically significant. A difference that is significant (p<0.05) might be too small to matter. A large effect might not be significant if sample size is small.

0 likes 19mo

r.marsdenTL3Regular2 Feb 2025#3

f.fontaine, post #2: Effect sizes: the magnitude of a difference, not just whether it is statistically significant. A difference that is significant (p Go to post

This follows post #2 rather than contradicting it.

For anyone arriving from a search: the marked solution above is the direct answer, and the replies underneath it add the caveats that make it safe to use.

6 likes in reply to #2 18mo

This topic was referenced in

Multiplicity when you track fifteen variablesResearch Methods › Statistics · 2 replies
Sample size intuition for a personal experimentResearch Methods › Statistics · 4 replies

Suggested topics

Topic	Participants	Replies	Views	Activity
Regression to the mean in progress reports Research Methods › Statistics number needed to treat heterogeneity confounding effect size On the subject in the title: Regression to the mean in progress reports Working notes rather than a conclusion. Session topic: PIONEER 6 ( N Engl J Med , 2019). Please read it before posting; the discussion…	+30	34	49k	12mo
[2026 update] Correlation in a self-tracked dataset: what it can support Research Methods › Statistics number needed to treat effect size worked example heterogeneity Posting this under the heading it deserves: Correlation in a self-tracked dataset: what it can support Everything below is what sits behind that. I have seen SURMOUNT-4 ( JAMA , 2024) cited in support of a…		2	31k	6h
What a confidence interval means, from scratch Research Methods › Statistics number needed to treat confounding worked example heterogeneity What a confidence interval means, from scratch — that is the question, and I have not found it answered plainly anywhere I have looked. I have seen STEP 8 ( JAMA , 2022) cited in support of a claim I do not…		2	2.6k	2mo
Multiplicity when you track fifteen variables Research Methods › Statistics effect size heterogeneity confounding worked example Solved Multiplicity when you track fifteen variables — setting out what I have, and where I think it stops being reliable. Comparing LEADER ( N Engl J Med , 2016) with SURMOUNT-OSA ( N Engl J Med , 2024) and finding…		2	60k	11mo
Measurement error in home scales, with a worked standard deviation — the long version Research Methods › Statistics number needed to treat worked example heterogeneity effect size Solved On the subject in the title: Measurement error in home scales, with a worked standard deviation — the long version Working notes rather than a conclusion. Session topic: SURMOUNT-1 ( N Engl J Med , 2022).…	+96	105	2k	2mo

Related topics — sharing the tags number needed to treat, confounding, effect size

Topic	Participants	Replies	Views	Activity
Fixed versus detachable needles: the trade-offs Devices › Syringes & needles dead volume subcutaneous injection syringe units worked example Solved Posting this under the heading it deserves: Fixed versus detachable needles: the trade-offs Everything below is what sits behind that. I have read the maintained page on this and I still have a gap, so I am…	+117	121	7.3k	18mo
Micro-titration: a disputed topic, argued properly Practice › Dosing & titration titration dose reduction worked example tirzepatide Solved Wiki Micro-titration: a disputed topic, argued properly Writing it up because I had to work it out twice and would rather nobody else did. A question about technique rather than about dose. I have been doing the…	+99	110	61k	2d
Journal club: SUSTAIN 6 and its retinopathy signal — does this still hold? Evidence › Journal club journal club effect size number needed to treat randomised trial The question in the title: Journal club: SUSTAIN 6 and its retinopathy signal — does this still hold? I will give what I have already checked below so nobody repeats it. I have seen SURMOUNT-2 ( Lancet ,…	+60	66	12k	16mo
Titles that describe the question, not the situation — what changed since Start Here › How to Ask beginner friendly site feedback worked example guidelines Posting this under the heading it deserves: Titles that describe the question, not the situation — what changed since Everything below is what sits behind that. Question in the title. Context below, and I…	+29	33	29k	10mo
Revisiting: The arithmetic of an intermediate dose between two label steps Practice › Dosing & titration titration worked example dose reduction tirzepatide Revisiting: The arithmetic of an intermediate dose between two label steps Writing it up because I had to work it out twice and would rather nobody else did. I have read the maintained page on this and I…	+41	45	808	2mo

Sponsored Janoshik Analytical — independent third-party testingHPLC purity and mass-spectrometric identity on submitted samples, reported with the method stated. The laboratory most often cited in this community’s verification records.Independent testing → Sponsored GL Biochem (Shanghai) Ltd. — synthesis, directFounded 1998. ISO 9001 and cGMP certified, over 1,500 staff and 200-plus patents. Custom peptide synthesis and amino acid derivatives, with a batch certificate on every order.Custom synthesis →