Pre-registration: the slow-moving methodological revolution

In 2015, the Open Science Framework's pre-registration system launched at scale. The pitch was simple: researchers commit, in advance, to specific hypotheses, sample sizes, and analysis methods. The commitments are recorded in a tamper-proof registry. When the study publishes, readers can verify that the reported analyses match the planned ones.

The mechanism was supposed to eliminate p-hacking, hypothesis-after-results-known (HARKing), and selective reporting — three of the largest contributors to the replication crisis.

A decade in, pre-registration has spread substantially and produced measurable changes in the literature. The picture is mostly positive, with some complications.

1. The growth

Pre-registration uptake has grown from roughly 100 pre-registered studies in 2015 to over 50,000 by 2024 across the OSF and similar platforms. Several major journals now require pre-registration for primary research, and the Registered Reports format — where journals commit to publish based on the design before data collection — has expanded from a handful of journals to hundreds (Nosek et al., 2018).

This is fast for academic culture change but slow as a fraction of total published research. Most psychology and most adjacent fields still publish substantial non-preregistered work.

2. What pre-registration actually changes

Studies comparing pre-registered to non-pre-registered work in matched contexts show:

Lower effect sizes in pre-registered studies. A consistent finding across reviews: pre-registered studies report smaller effects than equivalent non-pre-registered ones, by roughly 30-50% on average. The differences are usually attributable to the elimination of selective reporting (Kaplan & Irvin, 2015).

More null results. Pre-registered studies are roughly 3x more likely to report findings of "no significant effect" than non-pre-registered studies in the same domain. The honest fraction of null results in psychology is closer to 40-60% than the 10-20% that has historically been published.

Larger samples. Pre-registration tends to require explicit power analysis, which has pushed sample sizes upward.

These effects together suggest the pre-registered literature is more reliable than the non-pre-registered. The cost is that pre-registered work makes weaker claims with broader confidence intervals.

3. The unanticipated friction

Pre-registration has produced practical complications:

Exploratory analysis is harder to communicate. Many real insights come from unanticipated patterns in the data. Pre-registration constrains how these can be reported in primary analyses. Some researchers report pre-registered confirmatory and clearly-labeled exploratory analyses; others avoid exploratory work to maintain registered cleanliness.

Adversarial replication. Pre-registration makes it easier to identify when researchers deviated from their plans, which has produced public disputes that the field hadn't previously experienced.

Cost. Pre-registration adds friction to the research process — drafting plans, anticipating analytical choices, managing deviations. The fixed cost is real and discourages adoption in time-constrained contexts.

4. The reformed registered reports format

The most thorough version — Registered Reports (RR) — gets journal commitment to publish before data collection. Studies show:

• RR papers are significantly less likely to report positive findings than conventional papers
• RR papers replicate at higher rates in subsequent studies
• The proportion of RR submissions that reach final publication is high, suggesting the in-principle-acceptance commitment works (Allen & Mehler, 2019)

RR is the strongest format. It is also the slowest to adopt because both editors and authors have to commit time before knowing the result.

5. The honest summary

Pre-registration is one of the better-evidenced methodological reforms in modern science. The literature published under it is more reliable, on average, than the conventional literature. The reform has been partially successful — adoption is substantial but not universal — and the existing pre-2015 literature still contains substantial unreplicated work that hasn't been re-examined.

For a reader assessing claims from psychology in 2026: pre-registration status is a useful filter. Pre-registered studies should be given more weight per study than conventional studies of similar size. This is uncomfortable for the substantial backcatalog of important-seeming findings that were published before pre-registration was standard.