A study too small to detect an effect will sometimes detect one, and when it does the effect will be large.
This is not a paradox but a consequence of the at which results are published.
To reach with few participants, an estimate must be extreme.
Small studies therefore publish only their , and their underestimates stay in a drawer.
A literature built from small studies will accordingly report an average effect larger than the real one.
This is why large so often find a smaller effect and are described as failures.
They are not failures; they are the first reliable estimate anybody has produced.
The pattern has a visible that requires no access to unpublished work.
Plotting effect size against sample size should produce a , wide at the bottom and narrow at the top.
A missing one of its lower corners indicates that small studies finding nothing were not published.
The test is simple, it has been available for decades, and it is run on a minority of published reviews.
of studies before they begin addresses the problem at its source.
A registered study that never appears is a visible absence rather than an invisible one.
Where registration has been made a condition of publication, reported effect sizes have fallen substantially.
Nothing about the world changed; the sample of studies reaching print did.
Researchers sometimes object that small studies are all that a modest budget permits.
That is true and is an argument for , not an argument for publishing each one separately.
Ten small studies analysed together answer a question that ten small papers only appear to.