In 12 well-trained male cyclists who habitually take caffeine, 3 mg/kg of caffeine 90 min before a time trial of about an hour improved it whether or not the riders had first gone four days without caffeine (by 3.0% after the withdrawal and 3.6% without it), and the two caffeine trials did not differ significantly. From the abstract

Irwin and colleagues tested whether four days without caffeine change the benefit of a caffeine dose before an endurance test. Twelve well-trained, familiarised male cyclists who regularly took caffeine from coffee and other sources completed four trials in a double-blind, placebo-controlled crossover.

For four days they avoided all dietary caffeine and swallowed capsules each morning and afternoon holding either placebo or caffeine (1.5 mg/kg a day in total). On day five they took placebo or 3 mg/kg of caffeine 90 min before a time trial set at the work of one hour at 75% of peak sustainable power.

That gave four conditions: placebo-placebo, placebo-caffeine, caffeine-placebo and caffeine-caffeine. Acute caffeine shortened the time trial after the withdrawal and after the non-withdrawal period, and the two caffeine trials did not differ significantly. Mean heart rate was higher with acute caffeine; ratings of perceived exertion did not differ.

The authors conclude that 3 mg/kg improves performance whether or not habitual users first withdraw for four days.

n = 12 well-trained, familiarised male cyclists who habitually consumed caffeine from coffee and other sources. Age, fitness and habitual intake are not given in the abstract.