Research Methods · Statistics

What a confidence interval means, from scratch

Research Methods › Statistics number needed to treat confounding worked example heterogeneity

microgramsTL2Regular26 Apr 2026#1

What a confidence interval means, from scratch — that is the question, and I have not found it answered plainly anywhere I have looked.

I have seen STEP 8 (JAMA, 2022) cited in support of a claim I do not think it supports, twice this month, so I would like to work through what it actually shows.

My reading is that the trial is sound for its own question and is being stretched to answer a different one. I might be wrong about that, which is why this is a topic rather than a correction.

What I would like from this discussion: someone who disagrees with me to say why, with the section of the paper they are relying on.

0 likes 3mo

hana.satoTL4 Moderator17 May 2026#2

On the opening post — agreed on the reasoning, with one qualification.

Absence of evidence and evidence of absence: if a study is small and finds no effect, that is absence of evidence, not evidence of absence. A larger study might find an effect that a small study missed.

0 likes 2mo

n.okwuosaTL2 Moderator31 May 2026 · edited#3

micrograms, post #1: What a confidence interval means, from scratch — that is the question, and I have not found it answered plainly anywhere I have looked. I have seen STEP 8 ( JAMA , 2022) cited in support of a claim I do not think it supports, twice this month, so I would like to work through what it actually shows. My reading is that the trial is sound… Go to post

Picking up post #2: that is the part I would want checked first.

Confidence intervals: rather than a single point estimate, a range of plausible values. A narrow interval means precise measurement; a wide interval means measurement is imprecise. Wider intervals (more uncertainty) are honest about limitation.

9 likes in reply to #1 2mo

This topic was referenced in

Measurement error in home scales, with a worked standard deviation — a second datasetResearch Methods › Statistics · 2 replies

Suggested topics

Topic	Participants	Replies	Views	Activity
Measurement error in home scales, with a worked standard deviation Research Methods › Statistics effect size heterogeneity number needed to treat confounding Solved Wiki Measurement error in home scales, with a worked standard deviation Writing it up because I had to work it out twice and would rather nobody else did. Comparing SURPASS-4 ( Lancet , 2021) with STEP 1 ( N Engl…	+26	31	4.3k	4mo
Second pass at: Regression to the mean in progress reports Research Methods › Statistics effect size worked example heterogeneity Second pass at: Regression to the mean in progress reports Writing it up because I had to work it out twice and would rather nobody else did. Comparing SUSTAIN 6 ( N Engl J Med , 2016) with STEP 4 ( JAMA ,…	+71	75	4k	17mo
Multiplicity when you track fifteen variables Research Methods › Statistics effect size heterogeneity confounding worked example Solved Multiplicity when you track fifteen variables — setting out what I have, and where I think it stops being reliable. Comparing LEADER ( N Engl J Med , 2016) with SURMOUNT-OSA ( N Engl J Med , 2024) and finding…		2	60k	11mo
Sample size intuition for a personal experiment Research Methods › Statistics number needed to treat heterogeneity confounding effect size On the subject in the title: Sample size intuition for a personal experiment Working notes rather than a conclusion. Session topic: STEP 2 ( Lancet , 2021). Please read it before posting; the discussion is…		4	62k	18mo
Second pass at: Absence of evidence and evidence of absence Research Methods › Statistics worked example confounding number needed to treat On the subject in the title: Second pass at: Absence of evidence and evidence of absence Working notes rather than a conclusion. Session topic: SELECT ( N Engl J Med , 2023). Please read it before posting;…	+50	55	18k	16h

Related topics — sharing the tags number needed to treat, confounding, worked example

Topic	Participants	Replies	Views	Activity
When a network meta-analysis is defensible and when it is not Evidence › Meta-analyses effect size heterogeneity publication bias Solved When a network meta-analysis is defensible and when it is not I have a specific reason for asking rather than idle curiosity, and the context is below. Comparing SUSTAIN 6 ( N Engl J Med , 2016) with LEADER (…		2	27k	4h
Journal club: STEP 3 and the intensive behavioural therapy floor — does this still hold? Evidence › Journal club journal club effect size number needed to treat Solved The question in the title: Journal club: STEP 3 and the intensive behavioural therapy floor — does this still hold? I will give what I have already checked below so nobody repeats it. Session topic: STEP 4 (…		2	15k	10mo
Coming back to: A community side-effect dataset, with its response rate and biases Data & Tools › Datasets observational data site feedback confounding data table On the subject in the title: A community side-effect dataset, with its response rate and biases Working notes rather than a conclusion. Posting the method first, because I know what the first three replies…	+27	31	46k	9mo
Sample size intuition for a personal experiment Research Methods › Statistics number needed to treat heterogeneity confounding effect size On the subject in the title: Sample size intuition for a personal experiment Working notes rather than a conclusion. Session topic: STEP 2 ( Lancet , 2021). Please read it before posting; the discussion is…		4	62k	18mo
Journal club: PIONEER 6 as a safety trial Evidence › Journal club effect size journal club number needed to treat risk of bias Posting this under the heading it deserves: Journal club: PIONEER 6 as a safety trial Everything below is what sits behind that. Session topic: SURMOUNT-4 ( JAMA , 2024). Please read it before posting; the…	+14	18	262	5mo

Sponsored Janoshik Analytical — independent third-party testingHPLC purity and mass-spectrometric identity on submitted samples, reported with the method stated. The laboratory most often cited in this community’s verification records.Independent testing → Sponsored GL Biochem (Shanghai) Ltd. — synthesis, directFounded 1998. ISO 9001 and cGMP certified, over 1,500 staff and 200-plus patents. Custom peptide synthesis and amino acid derivatives, with a batch certificate on every order.Custom synthesis →