REF-1278 · Hopkins et al., 2001 · Sports Medicine · Paper · PubMed

How big must a test gain be to be real?

Lab power tests' typical error ~0.9-2.0% (threshold power ~1.5%), 1.3× more in non-athletes; a 2nd test runs ~1.2% better from practice. One 1-2% gain can be noise (B+, from the abstract).

Main caveat: Read from the abstract only. Study list, per-test numbers of studies, what "equivalent mean power" in a constant-power test denotes, and heterogeneity could…
moderate evidenceMeasured on non-athletes to highly trained athletes

Hopkins, Schabort and Hawley pooled the reliability of power (or work) in physical performance tests from 101 studies of healthy adults. Reliability was measured as the typical (standard) error of measurement expressed as a coefficient of variation (CV), and as the percent change in the mean between trials.

For a given exercise duration the most reliable measures were field tests of sprint running (CV about 0.9%), peak power in an incremental test on a treadmill or cycle ergometer (about 0.9%), equivalent mean power in a constant-power test lasting 1 minute to 3 hours on a treadmill or cycle ergometer (0.9 to 2.0%), lactate-threshold power (about 1.5%) and jump height or distance (about 2.0%); mean power on isokinetic ergometers was relatively unreliable (CV above 4%).

CVs were larger for non-athletes than athletes (1.3×), for female than male non-athletes (1.4×), for very short (about 1 s) and long (about 1 h) tests than for 1-min tests (up to 1.6×), and for respiratory- than ergometer-based measures of power (1.4 to 1.6×); the time between trials had no clear-cut effect.

In studies with more than two trials, the CV between the first two trials was 1.3 times that between later trials, and performance improved by 1.2% between the first two trials but by only 0.2% between later ones.

Our reading: on a good laboratory test a single retest that moves 1-2% is inside the noise; the second test of a series looks about 1% better from practice alone; and a "plateau" can only be judged against a band at least this wide.

Protocol

Design: meta-analytic review; n = 101 studies of healthy adults (the participant total is not in the abstract); outcome power or work.
Reliability statistics: typical (standard) error of measurement as a CV; percent change in the mean between trials.
Factors analysed: test type and duration; athletes vs non-athletes; sex (in non-athletes); respiratory- vs ergometer-based measures of power; time between trials; number of trials (the practice effect).
Not in the abstract: the search method and inclusion criteria, what "equivalent mean power" in a constant-power test denotes and how it was derived (for example from a time to exhaustion — not stated), the number of studies behind each figure, the meta-analytic model and heterogeneity.
Funding and interests: not stated in the abstract or the PubMed record.

Key Results

Measure (abstract)Typical error (CV)
Field test of sprint running~0.9%
Peak power, incremental test on a treadmill or cycle ergometer~0.9%
Equivalent mean power, constant-power test of 1 min to 3 h (treadmill or cycle)0.9 to 2.0%
Lactate-threshold power~1.5%
Jump height or distance~2.0%
Mean power, isokinetic ergometer> 4%
What makes a test noisier (CV ratio): non-athletes vs athletes 1.3×; female vs male non-athletes 1.4×; ~1-s and ~1-h vs 1-min tests up to 1.6×; respiratory- vs ergometer-based measures of power 1.4 to 1.6×. Time between trials: no clear-cut effect.
Practice effect (studies with more than two trials): the CV between the first two trials was 1.3 times the CV between later trials; performance improved by 1.2% between the first two trials and by 0.2% between later ones.

Mechanisms

Measured: test-retest typical error and the change in the mean between trials, pooled across studies.
Authors' interpretation: the figures should help practitioners and researchers choose or design good measures and protocols; a practice trial matters.
Inference: the difference between two tests carries about 1.4 times a single test's CV of noise (the square root of two, our arithmetic); with a CV of 1.5%, a 2% change on one retest is about one noise SD — fully compatible with no change.
Inference: the extra 1.2% on the second trial is learning (pacing, familiarity, effort), not fitness; a block review that compares a first-ever test with a second one will credit the training with it.
Inference: non-athletes pace and judge maximal effort less consistently, which fits their 1.3-times larger CV; the abstract gives no reason.

Practical Application

How big must a test gain be to be real? Lab power tests' typical error ~0.9-2.0% (threshold power ~1.5%), 1.3× more in non-athletes; a 2nd test runs ~1.2% better from practice. One 1-2% gain can be noise (B+, from the abstract).
Do one practice test before the baseline that counts: the first two trials differed more and by 1.2% on average; later trials by 0.2% (the authors' finding). How to repeat a test the same way each time: SYNTH-53.
Read block outcomes against a noise band. Not from this paper; the base's own practice (SYNTH-42): a change inside its noise band is no change. Bands in the base: this paper for laboratory tests, REF-752 (1.3-4.4% for a heart-rate-clamped submaximal test), REF-1279 (about 3% season to season for professionals' field critical power).
A trained athlete's expected gain is close to the noise. An interval block brings trained people about 2% in a time trial on average (REF-1275), which is close to a single retest's noise on this paper's figures if both are in the same unit (this paper's CVs are for power; REF-1275's abstract does not say whether its 2% is time or power) — so repeat the test, or judge across blocks (inference).
Field tests are likely noisier. Outdoor conditions, pacing without a set protocol and fatigue add variation that this laboratory pool does not include (inference; not tested here).

Limitations

Read from the abstract only. Study list, per-test numbers of studies, what "equivalent mean power" in a constant-power test denotes, and heterogeneity could not be checked.
Old protocols and equipment. Published in 2001; ergometers, power meters and test protocols have changed since.
Healthy adults, laboratory and field-sprint tests. Field FTP, ramp tests in apps and power from training files are not covered.
Search method not described in the abstract, so selection of the 101 studies cannot be judged.
Funding and interests: not stated in the abstract or the PubMed record.
How this entry was checked and changed
Why this level

pooled measurement data from 101 studies (a single reliability study sits at B in the rubric; pooling earns a step), held there because the search method is not described in the abstract, the protocols are laboratory and field tests up to 2001, and the number of studies behind each figure and the heterogeneity are not given; pre-2010, kept because it is still the base's only pooled estimate of endurance and power performance-test noise (REF-1166 pools isometric mid-thigh pull reliability). Read from the abstract (closed access; no Europe PMC copy, and the Victoria University repository record listed by OpenAlex is not open access and its host did not respond)