Research Methods · Statistics

Measurement error in home scales, with a worked standard deviation — a second dataset

Research Methods › Statistics effect size confounding heterogeneity number needed to treat

Solved

Solved by s.achebe in post #3

Regression to the mean: if you select people with extreme values (very high or very low), their next measurement is often less extreme just by chance. This can look like a treatment effect when it is just statistics.

Jump to the accepted answer →

a.kowalskiTL2 Moderator13 Jul 2025#1

Measurement error in home scales, with a worked standard deviation — a second dataset — setting out what I have, and where I think it stops being reliable.

Comparing FLOW (N Engl J Med, 2024) with SURPASS-2 (N Engl J Med, 2021) and finding the comparison harder than it looks.

Different populations, different durations, different endpoints defined slightly differently, and in one case a different estimand. People compare the headline percentages anyway, including me until recently.

Is there a defensible way to put these side by side, or is the honest answer that there is not and we should stop?

0 likes 13mo

KnowltonTL3Regular17 Oct 2025#2

Picking up the opening post: that is the part I would want checked first.

Power and sample size: a study might be too small to detect a real effect (low power). Sample size calculations help determine how many participants are needed to detect an effect of a given magnitude.

31 likes 9mo

s.achebeTL2 Moderator Solution25 Dec 2025#3

Regression to the mean: if you select people with extreme values (very high or very low), their next measurement is often less extreme just by chance. This can look like a treatment effect when it is just statistics.

16 likes 7mo

This topic was referenced in

Absence of evidence and evidence of absenceResearch Methods › Statistics · 20 replies

Suggested topics

Topic	Participants	Replies	Views	Activity
Second pass at: Absence of evidence and evidence of absence Research Methods › Statistics worked example confounding number needed to treat On the subject in the title: Second pass at: Absence of evidence and evidence of absence Working notes rather than a conclusion. Session topic: SELECT ( N Engl J Med , 2023). Please read it before posting;…	+50	55	18k	16h
Regression to the mean in progress reports Research Methods › Statistics number needed to treat heterogeneity confounding effect size On the subject in the title: Regression to the mean in progress reports Working notes rather than a conclusion. Session topic: PIONEER 6 ( N Engl J Med , 2019). Please read it before posting; the discussion…	+30	34	49k	12mo
[2026 update] Correlation in a self-tracked dataset: what it can support Research Methods › Statistics number needed to treat effect size worked example heterogeneity Posting this under the heading it deserves: Correlation in a self-tracked dataset: what it can support Everything below is what sits behind that. I have seen SURMOUNT-4 ( JAMA , 2024) cited in support of a…		2	31k	6h
What a confidence interval means, from scratch — the long version Research Methods › Statistics effect size worked example confounding number needed to treat The question in the title: What a confidence interval means, from scratch — the long version I will give what I have already checked below so nobody repeats it. I have seen SURMOUNT-2 ( Lancet , 2023) cited…	+16	20	2.3k	8mo
Second pass at: Regression to the mean in progress reports Research Methods › Statistics effect size worked example heterogeneity Second pass at: Regression to the mean in progress reports Writing it up because I had to work it out twice and would rather nobody else did. Comparing SUSTAIN 6 ( N Engl J Med , 2016) with STEP 4 ( JAMA ,…	+71	75	4k	17mo

Related topics — sharing the tags effect size, confounding, heterogeneity

Topic	Participants	Replies	Views	Activity
Sample size intuition for a personal experiment Research Methods › Statistics number needed to treat heterogeneity confounding effect size On the subject in the title: Sample size intuition for a personal experiment Working notes rather than a conclusion. Session topic: STEP 2 ( Lancet , 2021). Please read it before posting; the discussion is…		4	62k	18mo
Preprint servers and what screening they do — the long version Evidence › Preprints risk of bias effect size needs a source On the subject in the title: Preprint servers and what screening they do — the long version Working notes rather than a conclusion. Session topic: PIONEER 6 ( N Engl J Med , 2019). Please read it before…	+7	11	26k	4mo
Tirzepatide's shorter half-life and its one practical consequence Compounds › Tirzepatide SURPASS programme randomised trial effect size tirzepatide Wiki On the subject in the title: Tirzepatide's shorter half-life and its one practical consequence Working notes rather than a conclusion. I have seen SELECT ( N Engl J Med , 2023) cited in support of a claim I…	+18	29	20k	8mo
A pooled estimate that changed when one trial was added Evidence › Meta-analyses meta-analysis publication bias effect size Solved A pooled estimate that changed when one trial was added — setting out what I have, and where I think it stops being reliable. Session topic: STEP 4 ( JAMA , 2021). Please read it before posting; the…	+24	28	3.5k	2d
Journal club: SUSTAIN 6 and its retinopathy signal — does this still hold? Evidence › Journal club journal club effect size number needed to treat randomised trial The question in the title: Journal club: SUSTAIN 6 and its retinopathy signal — does this still hold? I will give what I have already checked below so nobody repeats it. I have seen SURMOUNT-2 ( Lancet ,…	+60	66	12k	16mo

Sponsored Sigma-Aldrich — certified reference materialsReference standards, analytical reagents and certified materials for method calibration. If you are validating a method, this is where the standard comes from.Reference standards →