Blinding yourself: practical methods and their limits

On the subject in the title: Blinding yourself: practical methods and their limits Working notes rather than a conclusion.

Comparing PIONEER 6 (N Engl J Med, 2019) with STEP 4 (JAMA, 2021) and finding the comparison harder than it looks.

Different populations, different durations, different endpoints defined slightly differently, and in one case a different estimand. People compare the headline percentages anyway, including me until recently.

Is there a defensible way to put these side by side, or is the honest answer that there is not and we should stop?

Practical note that does not fit anywhere else. Whatever you conclude from this topic, write down what you did and when. The single most useful thing in your own records is not any individual result; it is that they are dated and consecutive.

post #2 is right about the mechanism and I think understates the practical bit.

Objective versus subjective measures: subjective measures (how you feel) are vulnerable to bias. Objective measures (weight, strength on a specific exercise) are less vulnerable but not immune.

Worth separating two things that post #3 runs together.

Sample size in n-of-1: you are the sample. Repeated measurements (weekly weighings, daily mood scores) increase the power to detect a real effect even though n=1.

Picking up post #2: that is the part I would want checked first.

Thank you for the correction. I have edited my earlier post with a note rather than silently, so the thread still makes sense to read. The error was mine and it was the kind that comes from remembering a figure instead of looking it up.

Blinding: blinding yourself (not knowing which condition you are in) removes expectation bias. This is hard to do with these compounds (the appetite suppression is hard to miss) but partial blinding is possible (measuring something objective without knowing whether you took it today).

Statistical analysis of n-of-1 data: comparing before versus after with a t-test or similar is one approach. Plotting the data visually is another. Both are valid.

On post #4 — agreed on the reasoning, with one qualification.

Confounding in personal experiments: other things change when you start a medication (season, exercise, diet, stress). Documenting those confounders helps you understand their contribution to the result.

This follows post #6 rather than contradicting it.

Generalisability: a robust n-of-1 result applies to you. It does not tell you much about whether the effect generalises to others similar to you, much less to people different from you.

I read post #8 twice before replying, because I had assumed the opposite.

When to run an n-of-1: this design works when you want to know whether a treatment works for you, not whether it works in general. For that purpose, it is efficient.

Worth separating two things that post #7 runs together.

Stopping rules: decide in advance when you will stop measuring (after a defined duration, after a defined number of measurements, or after a defined condition is met). Not deciding in advance means stopping when the result satisfies you, which is bias.

post #11 is right about the mechanism and I think understates the practical bit.

Designing a personal experiment that could actually change your mind: that is the standard for an n-of-1 design. An experiment designed so that any result confirms what you already believed has not changed anything.

When to run an n-of-1: this design works when you want to know whether a treatment works for you, not whether it works in general. For that purpose, it is efficient.

Thank you for the correction. I have edited my earlier post with a note rather than silently, so the thread still makes sense to read. The error was mine and it was the kind that comes from remembering a figure instead of looking it up.