In much the same way that everything in your fridge both causes and prevents cancer, Alex Hutchinson notes, there’s a study out there somewhere proving that everything boosts endurance:
A new preprint (a journal article that hasn’t yet been peer-reviewed, ironically) from researchers at Queensland University of Technology in Australia explores why this seems to be the case, and what can be done about it. David Borg and his colleagues comb through thousands of articles from 18 journals that focus on sport and exercise medicine, and unearth telltale patterns about what gets published—and perhaps more importantly, what doesn’t. To make sense of the studies you see and decide whether the latest hot performance aid is worth experimenting with, you also have to consider the studies you don’t see.
[...]
One way to illustrate these results is to plot something called the z-value, which is a statistical measure of the strength of an effect. In theory, if you plot the z-values of thousands of studies, you’d expect to see a perfect bell curve. Most of the results would be clustered around zero, and progressively fewer would have either very strongly positive or very strongly negative effects. Any z-value less than -1.96 or greater than +1.96 corresponds to a statistically significant result with p less than 0.05. A z-value between -1.96 and +1.96 indicates a null result with no statistically significant finding.
In practice, the bell curve won’t be perfect, but you’d still expect a fairly smooth curve. Instead, this is what you see if you plot the z-values from the 1,599 studies analyzed by Borg:
There’s a giant missing piece in the middle of the bell curve, where all the studies with non-significant results should be. There are probably lots of different reasons for this, both driven by decisions that researchers make and—just as importantly—decisions that journals make about what to publish and what to reject. It’s not an easy problem to solve, because no journal wants to publish (and no reader wants to read) thousands of studies that conclude, over and over, “We’re not yet sure whether this works.”