benchmark.js Allow Asynchronous Calls in setup()/teardown() Functions

This pull request is in reference to issue #70. To start off, this is a breaking change. When the defer option is set to true for a benchmark, both the setup() and teardown() functions require their resolve functions be called (suResolve() and tdResolve() respectively). If the functions are not included with the benchmark, the resolve calls are generated allowing setup() and teardown() functions to be omitted.

I tried to maintain the coding style as you describe, but have no problem changing variable names and such if you feel something else works better.

Currently, the unit tests are updated and are all passing when run with Node.js v4.x. I'm not very versed in running with the browser, so I'd appreciate any assistance people are willing to provide to test out these changes more. This includes verifying that the accuracy of the benchmarks has not been compromised.

Please reference the unit tests for examples of use. Documentation will need to be updated, but wanted to get some validation these changes will be accepted before investing the time to document.

Thanks for all the work you've put into benchmark.js.

Nov 04 '16 22:11 meanjoe45

I have run the unit tests with Node.js v4.4.2 and v6.9.1 locally on my machine and everything passes. I'm not sure yet why the CI build failed. I'll try to investigate soon, but help is appreciated.

Nov 05 '16 16:11 meanjoe45

@jdalton can we get this merged? this really a required feature where there is no option except async setup / teardown

Dec 27 '16 22:12 shamasis

Is there a reason this has not been merged yet?

Mar 16 '18 14:03 daprahamian

@jdalton is there anything that can be done to merge this? This is a very useful patch for doing async setup/teardown. Do you want to consider changing it so the breaking nature of the change doesn't occur?

Apr 24 '18 01:04 gwynjudd

@jdalton ping :-|

Jun 14 '18 04:06 shamasis

+1 This feature is really awesome and I really need (I think a lot of people too). Waiting for release.

Aug 09 '18 12:08 vflopes

We are trying to use benchmark.js for benchmarking the JS interface to some logic implemented in wasm and are also stuck because our wasm instantiation needs to happen asynchronously. Having async setup would definitely be helpful!

Oct 09 '18 03:10 rajsite

Just bumping this as I ran into it today given the last movement was ~ a year ago. Is there anything that needs to change for this PR? I'm happy to try to pitch in.