The Peptide CommonsEst. May 2024
Independent. We sell nothing and are affiliated with no manufacturer or pharmacy. Every moderation action is logged in public
Evidence · Trials

Adjudicated events and why the definition matters — the long version

JS
j.steinerTL2 Moderator2 Feb 2025#1

Adjudicated events and why the definition matters — the long version Writing it up because I had to work it out twice and would rather nobody else did.

Comparing SURPASS-4 (Lancet, 2021) with STEP 4 (JAMA, 2021) and finding the comparison harder than it looks.

Different populations, different durations, different endpoints defined slightly differently, and in one case a different estimand. People compare the headline percentages anyway, including me until recently.

Is there a defensible way to put these side by side, or is the honest answer that there is not and we should stop?

5 likes 18mo
EF
e.ferreiraTL3Regular4 Feb 2025#2

the opening post answers the question as asked. The question underneath it is different.

Confounding in observational data: a third variable can explain an apparent association. In a randomised trial, randomisation balances unknown confounders. In observational data, observed confounders can be adjusted for but unknown ones cannot.

0 likes 18mo
IB
i.boatengTL2 Moderator6 Feb 2025#3

Dropout is information: high dropout rates can indicate tolerability problems or lower efficacy than the summary suggests. Where the analysis handled dropouts matters. An intention-to-treat analysis with many dropouts can give a smaller apparent effect than per-protocol analysis.

20 likes 18mo
SS
steady_stateTL3Regular7 Feb 2025#4

Open-label design: unblinded trials admit expectation effects. For weight-loss trials where one arm loses substantial weight and the other does not, complete blinding is impossible anyway. The unblinded nature is a limitation worth noting.

9 likes 18mo
RE
r.erdoganTL2 Moderator8 Feb 2025#5

Worth separating two things that the opening post runs together.

Generalisability: the enrolled population was selected in ways that matter. Entry criteria, run-in periods, and the simple fact that people who agree to a multi-year trial differ from people who do not, all narrow the population. That is how internal validity is bought, at the cost of external validity.

5 likes 18mo
NA
n.abernathyTL3Analytical chemist9 Feb 2025#6
r.erdogan, post #5: Worth separating two things that the opening post runs together. Generalisability: the enrolled population was selected in ways that matter. Entry criteria, run-in periods, and the simple fact that people who agree to a multi-year trial differ from people who do not, all narrow the population. That is how internal validity is bought, at… Go to post

post #5 is right about the mechanism and I think understates the practical bit.

Population narrowness: most trials in this class enrolled fairly specific groups. Baseline body mass index ranges, exclusion of renal disease, exclusion of certain comorbidities, all narrow the population. Applying point estimates to someone well outside the range is an extrapolation.

0 likes in reply to #5 18mo
BK
b.kowalskiTL2 Moderator11 Feb 2025 · edited#7

Having read the exchange above, I think I was wrong earlier in this topic and I want to say so plainly rather than quietly editing.

The correction was fair and I had been repeating something I had not checked carefully enough.

28 likes 18mo
DH
dietitian_hollisTL3Dietitian12 Feb 2025#8

Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.

13 likes 17mo
AV
a.vukovicTL2 Moderator13 Feb 2025#9
steady_state, post #4: Open-label design: unblinded trials admit expectation effects. For weight-loss trials where one arm loses substantial weight and the other does not, complete blinding is impossible anyway. The unblinded nature is a limitation worth noting. Go to post

On post #5 — agreed on the reasoning, with one qualification.

Two things before anyone answers the substance.

First, the context in the first post is clear and specific. Second, the question is framed so that an answer can actually address it. Both are the norm here and both matter more than they sound.

8 likes in reply to #4 17mo
BN
bench_notesTL4 Moderator14 Feb 2025#10

Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.

2 likes 17mo
EK
ew.kuuselaTL2 Moderator14 Feb 2025#11
e.ferreira, post #2: the opening post answers the question as asked. The question underneath it is different. Confounding in observational data: a third variable can explain an apparent association. In a randomised trial, randomisation balances unknown confounders. In observational data, observed confounders can be adjusted for but unknown ones cannot. Go to post

The estimand: what the trial set out to estimate. Two trials can be identical in structure but estimate different things by using different handling rules for people who stop taking the drug. Treatment-policy and hypothetical approaches are both legitimate but answer different questions.

27 likes in reply to #2 17mo
BT
baseline_tableTL2Member15 Feb 2025#12
r.erdogan, post #5: Worth separating two things that the opening post runs together. Generalisability: the enrolled population was selected in ways that matter. Entry criteria, run-in periods, and the simple fact that people who agree to a multi-year trial differ from people who do not, all narrow the population. That is how internal validity is bought, at… Go to post

Multiplicity and multiple comparisons: if a trial tests many hypotheses, the chance of a false positive on at least one by random chance increases. This is why pre-specification of the primary endpoint matters and why secondary endpoints are weaker evidence.

0 likes in reply to #5 17mo
TB
t.brandtTL2 Moderator16 Feb 2025#13

This follows post #10 rather than contradicting it.

Intent-to-treat versus per-protocol: ITT includes everyone assigned regardless of whether they took the drug. Per-protocol includes only those who completed it as intended. The two can give substantially different results.

4 likes 17mo
KB
k.bettencourtTL2Member17 Feb 2025#14

I read post #12 twice before replying, because I had assumed the opposite.

Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.

13 likes 17mo
JL
j.lokkenTL2 Moderator18 Feb 2025#15
e.ferreira, post #2: the opening post answers the question as asked. The question underneath it is different. Confounding in observational data: a third variable can explain an apparent association. In a randomised trial, randomisation balances unknown confounders. In observational data, observed confounders can be adjusted for but unknown ones cannot. Go to post

post #14 answers the question as asked. The question underneath it is different.

Confounding in observational data: a third variable can explain an apparent association. In a randomised trial, randomisation balances unknown confounders. In observational data, observed confounders can be adjusted for but unknown ones cannot.

0 likes in reply to #2 17mo
SS
s.stavrianosTL2Member19 Feb 2025#16

Two things before anyone answers the substance.

First, the context in the first post is clear and specific. Second, the question is framed so that an answer can actually address it. Both are the norm here and both matter more than they sound.

0 likes 17mo
AV
a.villalobosTL2 Moderator20 Feb 2025 · edited#17

Intent-to-treat versus per-protocol: ITT includes everyone assigned regardless of whether they took the drug. Per-protocol includes only those who completed it as intended. The two can give substantially different results.

8 likes 17mo
D
DSakamotoTL3Regular21 Feb 2025#18

Coming back to post #16, because the follow-up matters more than the original answer.

Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.

19 likes 17mo
CF
c.falkTL2 Moderator21 Feb 2025#19
DSakamoto, post #18: Coming back to post #16, because the follow-up matters more than the original answer. Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or… Go to post

post #18 is right about the mechanism and I think understates the practical bit.

Thank you for the correction. I have edited my earlier post with a note rather than silently, so the thread still makes sense to read. The error was mine and it was the kind that comes from remembering a figure instead of looking it up.

14 likes in reply to #18 17mo
AW
a.westergaardTL3Regular22 Feb 2025#20

Worth separating two things that post #16 runs together.

Multiplicity and multiple comparisons: if a trial tests many hypotheses, the chance of a false positive on at least one by random chance increases. This is why pre-specification of the primary endpoint matters and why secondary endpoints are weaker evidence.

28 likes 17mo
IW
i.wojcikTL2 Moderator23 Feb 2025#21

The estimand: what the trial set out to estimate. Two trials can be identical in structure but estimate different things by using different handling rules for people who stop taking the drug. Treatment-policy and hypothetical approaches are both legitimate but answer different questions.

23 likes 17mo
V
VPoulsenTL3Regular24 Feb 2025#22

Generalisability: the enrolled population was selected in ways that matter. Entry criteria, run-in periods, and the simple fact that people who agree to a multi-year trial differ from people who do not, all narrow the population. That is how internal validity is bought, at the cost of external validity.

10 likes 17mo
BF
b.friskTL2 Moderator25 Feb 2025#23
i.wojcik, post #21: The estimand: what the trial set out to estimate. Two trials can be identical in structure but estimate different things by using different handling rules for people who stop taking the drug. Treatment-policy and hypothetical approaches are both legitimate but answer different questions. Go to post

Coming back to post #21, because the follow-up matters more than the original answer.

Having read the exchange above, I think I was wrong earlier in this topic and I want to say so plainly rather than quietly editing.

The correction was fair and I had been repeating something I had not checked carefully enough.

3 likes in reply to #21 17mo
BE
bench_entryTL3Regular25 Feb 2025#24
s.stavrianos, post #16: Two things before anyone answers the substance. First, the context in the first post is clear and specific. Second, the question is framed so that an answer can actually address it. Both are the norm here and both matter more than they sound. Go to post

Picking up post #21: that is the part I would want checked first.

Population narrowness: most trials in this class enrolled fairly specific groups. Baseline body mass index ranges, exclusion of renal disease, exclusion of certain comorbidities, all narrow the population. Applying point estimates to someone well outside the range is an extrapolation.

0 likes in reply to #16 17mo
KA
k.asanteTL2 Moderator26 Feb 2025 · edited#25

Open-label design: unblinded trials admit expectation effects. For weight-loss trials where one arm loses substantial weight and the other does not, complete blinding is impossible anyway. The unblinded nature is a limitation worth noting.

30 likes 17mo
CO
c.okaforTL3Regular27 Feb 2025#26

post #25 is right about the mechanism and I think understates the practical bit.

Surrogate endpoints: an endpoint that is not the outcome that matters but is measured as a stand-in. HbA1c is a surrogate for long-term glucose control and the short-term complications it prevents. Weight loss is a surrogate for metabolic health and long-term outcomes. Surrogates are useful but not identical to the endpoint that matters.

15 likes 17mo
ND
n.duarteTL2 Moderator28 Feb 2025#27

I read post #25 twice before replying, because I had assumed the opposite.

Dropout is information: high dropout rates can indicate tolerability problems or lower efficacy than the summary suggests. Where the analysis handled dropouts matters. An intention-to-treat analysis with many dropouts can give a smaller apparent effect than per-protocol analysis.

6 likes 17mo
CC
crossref_checkTL3Wiki editor1 Mar 2025#28
VPoulsen, post #22: Generalisability: the enrolled population was selected in ways that matter. Entry criteria, run-in periods, and the simple fact that people who agree to a multi-year trial differ from people who do not, all narrow the population. That is how internal validity is bought, at the cost of external validity. Go to post

Thank you for the correction. I have edited my earlier post with a note rather than silently, so the thread still makes sense to read. The error was mine and it was the kind that comes from remembering a figure instead of looking it up.

1 like in reply to #22 17mo
PO
p.ostergaardTL2 Moderator1 Mar 2025#29

On post #25 — agreed on the reasoning, with one qualification.

Two things before anyone answers the substance.

First, the context in the first post is clear and specific. Second, the question is framed so that an answer can actually address it. Both are the norm here and both matter more than they sound.

0 likes 17mo
CL
customs_ledgerTL3Regular2 Mar 2025#30

Generalisability: the enrolled population was selected in ways that matter. Entry criteria, run-in periods, and the simple fact that people who agree to a multi-year trial differ from people who do not, all narrow the population. That is how internal validity is bought, at the cost of external validity.

22 likes 17mo