Benchmarked function is run one more time than specified

[flenzen]

Consider the following example:

first_run = false
function f()
  global first_run
  if !first_run; first_run = true; else sleep(1); end
  println(time())
end
@benchmark f() evals=1 samples=1 

which outputs
1.734518515199018e9
1.734518516201369e9
BenchmarkTools.Trial: 1 sample with 1 evaluation.
Single result which took 1.002 s (0.00% GC) to evaluate,
with a memory estimate of 624 bytes, over 11 allocations.

It seems that `f` is run an additional time before the actual sample is taken. This seems not to be documented. I wonder if this is to even compilation time out. If yes, where do I find information about that? In particular, is it possible to skip this, or trigger compilation with a cheaper instance?

Run on Julia 1.10 with BenchmarkTools v1.5.0.

[willow-ahrens]

This behavior is expected. There is some mention of warmups in the documentation, but I'm wondering whether you see a place where the docs state that there is no warmup? You can pass the warmup=false option to run if you would like to skip the warmup. Does this resolve your issue? Please let me know if there is a specific place in the docs that seems incorrect, misleading, or incomplete.