A structured critique template this community uses — a second dataset

On the subject in the title: A structured critique template this community uses — a second dataset Working notes rather than a conclusion.

Session topic: SELECT (N Engl J Med, 2023). Please read it before posting; the discussion is much better when everyone has.

The question I would like us to start with is what the trial set out to estimate, rather than what it found. Once that is on the table we can talk about whether the design could have answered it, and only then about the numbers.

Specific things I would like covered: the population and how far it generalises, how discontinuation was handled, whether the comparator was a fair one, and what the absolute rather than relative effect looks like.

I will summarise at the end and the summary will feed the relevant digest page.

Practical note that does not fit anywhere else. Whatever you conclude from this topic, write down what you did and when. The single most useful thing in your own records is not any individual result; it is that they are dated and consecutive.

post #15 answers the question as asked. The question underneath it is different.

Choosing the worst interpretation: "The confidence interval includes a harmful effect" is true if the CI goes from -1 to +5. But assuming the worst-case scenario is not how you use the evidence. The point estimate and the precision both matter.

Worth separating two things that post #29 runs together.

Choosing the worst interpretation: "The confidence interval includes a harmful effect" is true if the CI goes from -1 to +5. But assuming the worst-case scenario is not how you use the evidence. The point estimate and the precision both matter.

Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing.

post #42 answers the question as asked. The question underneath it is different.

Publication bias: a single published positive trial is weaker evidence than multiple published trials with consistent results. Asking whether there are unpublished negative trials is a fair critical question.