Setup function not run per sample, but per batch

[webstrand]

I was confused by the documentation and assumed the setup function would run prior to each run of the test function. But instead it runs per batch, which could be of variable size, so I can't prepare n objects for the test function to operate on.

Maybe something like this would be effective?

export function getDiff(cycles:number, callback:(setupData?:unknown)=>void, setup?:()=>unknown) {
    if(setup) {
        const setupData = Array.from({ length: cycles }, () => setup());
        const startTS = performance.now();
        while(cycles-- > 0) {
            callback(setupData[cycles]);
        }
        return performance.now() - startTS;
    }
    const startTS = performance.now();
    while(cycles-- > 0) {
        callback();
    }
    return performance.now() - startTS;
}

Since you don't want to include the setup function in the time measurement, and to avoid introducing extra error you need to avoid calling performance.now at the beginning and end of every batch.

Or I suppose you could pass the count into the setup function and the cycle into the test function, and let the user take care of building and selecting from the array?

If nothing else, making the documentation clearer would be nice.

The setup function to run on process creation, before any tests are run.

perhaps?

I really appreciate your library, thank you for releasing it.