Selection into a registry and what it does to the estimate

Selection into a registry and what it does to the estimate — setting out what I have, and where I think it stops being reliable.

Session topic: STEP 4 (JAMA, 2021). Please read it before posting; the discussion is much better when everyone has.

The question I would like us to start with is what the trial set out to estimate, rather than what it found. Once that is on the table we can talk about whether the design could have answered it, and only then about the numbers.

Specific things I would like covered: the population and how far it generalises, how discontinuation was handled, whether the comparator was a fair one, and what the absolute rather than relative effect looks like.

I will summarise at the end and the summary will feed the relevant digest page.

This follows the opening post rather than contradicting it.

For anyone arriving from a search: the marked solution above is the direct answer, and the replies underneath it add the caveats that make it safe to use.

Worth separating two things that the opening post runs together.

Multiple comparisons: if a paper reports many outcomes, the chance of a spurious association by random chance is real. Pre-specification of primary outcomes matters and secondary analyses are weaker evidence.