REF-1361 · Hopkins, 2000 · Sports Medicine · Paper · PubMed

How do I know a test result really changed?

Compare the change with the test's typical error, the spread of one person's repeat tests, best as a % of the mean; estimating it well needs ~50 people and 3+ trials (methods review).

Main caveat: Read from the abstract only. Worked examples, formulas and the author's recommendations for particular situations could not be checked.
moderate evidenceMeasured on a mixed group

Hopkins defines reliability as the reproducibility of a test's values in repeated trials on the same individuals: better reliability means more precise single measurements and better tracking of change. He names three main measures.

Within-subject random variation is best captured by the typical (standard) error of measurement — the standard deviation of an individual's repeated measurements — which for many measures in sports medicine and science is best expressed as a coefficient of variation (a percentage of the mean).

Limits of agreement, the 95% likely range of change in an individual's measurements between two trials, are a biased and more limited form of within-subject variation. Systematic changes in the mean between consecutive trials represent learning, motivation or fatigue and must be removed from estimates of within-subject variation.

The retest correlation is hard to interpret because its value depends on how heterogeneous the sample is. Reliability is used for decisions when monitoring individuals, for comparing tests or equipment, for estimating sample size, and for estimating the magnitude of individual differences in the response to a treatment.

Reasonable precision for a reliability estimate requires about 50 participants and at least 3 trials, and studies comparing reliability between tests or equipment need complex designs that are seldom analysed correctly. He argues for the typical error as the standard measure.

Our reading: this is the method behind the base's noise bands — the typical-error figures for performance tests come from the same author's later pooled analysis (REF-1278), and the rule for responders (an exercise arm's spread of change must exceed a control arm's) is built on the same quantity (REF-430, REF-1274, REF-1362).

Protocol

Design: methodological review; no data, no search.
Studies and participants: n = not applicable (the abstract reports no data; the "about 50 participants and at least 3 trials" is a recommendation for reliability studies).
Topics (abstract): definitions; typical error and coefficient of variation; limits of agreement; change in the mean; retest correlation; uses of reliability; sample size and design of reliability studies.
Funding and interests: not available from the abstract or the PubMed record.

Key Results

Question (abstract)What the review states
The standard measure of a test's noisetypical (standard) error of measurement — the SD of an individual's repeated measurements; usually best as a coefficient of variation (% of the mean)
Limits of agreementthe 95% likely range of change between 2 trials — a biased, more limited form of within-subject variation
A systematic change in the mean between trialslearning, motivation or fatigue — to be eliminated from estimates of within-subject variation
Retest correlationdifficult to interpret, sensitive to the heterogeneity of the sample
Usesdecisions when monitoring individuals; comparing tests or equipment; sample size; the magnitude of individual differences in response to a treatment
Precision of a reliability studyabout 50 participants and at least 3 trials
Comparing reliability between tests or devicesneeds complex designs and analyses that are "seldom" done correctly
Typical-error values for particular testsnot in the abstract (pooled values: REF-1278)

Mechanisms

Measured: nothing in the abstract; it is a statement of method.
Authors' interpretation: better reliability means more precise single measurements and better tracking of change, so the typical error is the quantity to use when deciding about an individual; the retest correlation is a poor guide because it moves with the heterogeneity of the sample.
Inference: in a varied sample a high retest correlation can coexist with a typical error too large for one person's change to be read.
Inference: the difference between two measurements carries about 1.4 times the typical error (the square root of two), which is why a change no larger than about one typical error cannot be told from no change (SYNTH-93's rule), and a change just beyond it passes only a roughly 50% interval (REF-1362); the √2 is stated in the full text of REF-1362, not in this abstract.
Inference: the first-trial learning effect the abstract tells researchers to remove is the 1.2% that the same author later measured between the first two trials of performance tests (REF-1278); in practice this is the familiarisation trial before a baseline that counts (SYNTH-53).

Practical Application

How do I know a test result really changed? Compare the change with the test's typical error, the spread of one person's repeat tests, best as a % of the mean; estimating it well needs ~50 people and 3+ trials (methods review).
Ask for the typical error, not the correlation: a device or test advertised with a high test-retest correlation can still be noisy for one person, because the correlation depends on how varied the sample was (B+, methods review, from the abstract).
Do a familiarisation trial: learning between the first trials shifts the mean and must be kept out of the noise estimate (the authors' rule); in performance tests the second trial ran about 1.2% better than the first (REF-1278). How to repeat a test the same way each time: SYNTH-53.
The athlete's own repeats beat a published figure: Not from this paper; the base's own practice: where the same effort has been repeated under the same conditions, the athlete's own typical error replaces the group figures (SYNTH-93).
What it is good for as a source: the definitions and the reasons behind them. What it does not show (in the abstract): the noise of any particular test or device; an individual's response.

Limitations

Read from the abstract only. Worked examples, formulas and the author's recommendations for particular situations could not be checked.
No data. The paper defines and argues; its level is that of a method source.
From 2000. The individual-response statistics the base uses were developed later on the same quantities (REF-430, REF-1274, REF-1362).
Limits of agreement. "Biased, more limited" is the author's judgement; limits of agreement remain widely reported, so a device study may give them instead of a typical error.
Funding and interests: not available from the abstract or the PubMed record. No device maker, federation or data platform is named.
How this entry was checked and changed
Why this level

a methodological review by the author who set the field's standard for the typical error, like the base's other methods sources on individual response (REF-430, REF-441); it reports no data of its own, so its level is that of an authoritative method statement, not of an effect. Read from the abstract (closed access; Europe PMC holds no full text and OpenAlex marks it closed; the Victoria University repository record it lists is not marked open and its host did not respond)