Issues with `BenchmarkTools.DEFAULT_PARAMETERS.samples` and `BenchmarkTools.DEFAULT_PARAMETERS.seconds`

[RoyiAvital]

Have a look at the following code:

using BenchmarkTools;
using Random;

BenchmarkTools.DEFAULT_PARAMETERS.samples   = 50;
BenchmarkTools.DEFAULT_PARAMETERS.evals     = 1;
BenchmarkTools.DEFAULT_PARAMETERS.seconds   = 15;

vX = rand(5_000);

startTime = time();
sBenchMark = @benchmark sin.(vY) setup = (vY = copy($vX));
runTime = time() - startTime;

display(sBenchMark);

println("\nTotal Run Time $runTime [Sec]\n");

On my computer each evaluation takes ~ 35 [Micro Sec]. Yet the run time is ~15 [Sec] even though it should do only 50 evaluations (~1750 Micro Sec).

[RoyiAvital]

By the way, it seems that manual settings works correctly:

sBenchMark = @benchmarkable sin.(vY) setup = (vY = copy($vX)) seconds = 1 samples = 50 evals = 1;
startTime = time();
sBenchMarkRes = run(sBenchMark);
runTime = time() - startTime;

display(sBenchMarkRes);

println("\nTotal Run Time $runTime [Sec]\n");

Which yields:

BenchmarkTools.Trial: 50 samples with 1 evaluation.
 Range (min … max):  29.400 μs …  33.300 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     30.100 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   30.320 μs ± 852.128 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

 Memory estimate: 39.14 KiB, allocs estimate: 2.

Total Run Time 0.2569999694824219 [Sec]

The whole file is given by: BenchMarkIssue.txt.