Preprint servers and what screening they do — the long version

On the subject in the title: Preprint servers and what screening they do — the long version Working notes rather than a conclusion.

Session topic: PIONEER 6 (N Engl J Med, 2019). Please read it before posting; the discussion is much better when everyone has.

The question I would like us to start with is what the trial set out to estimate, rather than what it found. Once that is on the table we can talk about whether the design could have answered it, and only then about the numbers.

Specific things I would like covered: the population and how far it generalises, how discontinuation was handled, whether the comparator was a fair one, and what the absolute rather than relative effect looks like.

I will summarise at the end and the summary will feed the relevant digest page.

Server repositories: medRxiv for medicine, bioRxiv for biology. Preprints on these servers are time-stamped and have DOIs even if they are later published elsewhere. Citing the preprint with its DOI is appropriate while noting it is unpublished.

post #2 is right about the mechanism and I think understates the practical bit.

Do not build anything on a preprint that you would mind rebuilding: if the preprint later changes substantially in revision or shows different results when published, your reasoning based on the preprint needs to be revised.

Practical note that does not fit anywhere else. Whatever you conclude from this topic, write down what you did and when. The single most useful thing in your own records is not any individual result; it is that they are dated and consecutive.

Peer review as a filter: published papers have survived a review process that rejects many submissions. Preprints have not. Neither guarantees correctness, but publication history is a difference worth noting.

How to cite a preprint: author last name, first initial, et al. Title. Server name [preprint]. Posted date. Doi. Include the server name and note that it is a preprint.

post #6 answers the question as asked. The question underneath it is different.

I disagree with the reply above, and I think the disagreement is substantive rather than terminological.

The distinction being drawn does not survive when you look at the published data for this specific question. I would be glad to be shown wrong on this, because the version I am arguing against is more convenient.

On post #4 — agreed on the reasoning, with one qualification.

Unrefereed work is not worthless. A great deal of it survives review substantially unchanged. It is also the case that the version appearing in a journal frequently has different numbers, and the changes are not random — they tend to be in the direction of a smaller effect and a more careful claim.

This follows post #6 rather than contradicting it.

Always label it as a preprint in any discussion. A preprint is not a published trial and it is not settled knowledge. Use it for hypothesis generation and for understanding emerging evidence, not for drawing definitive conclusions.

Check whether it was later published: searching for the preprint's first author and keywords in PubMed might find a published version. The published version is the stronger citation if it exists.

Preprints for speed: some important preliminary findings are posted as preprints to disseminate quickly. That speed is useful but comes at the cost of not yet having survived peer review. The trade-off is real.

Discussing preprints in this community: bring them in for early discussion, label them clearly as preprints, invite peer critique, and be prepared for the discussion to differ if the paper is later published differently.