benchmark
benchmark copied to clipboard
Benchmark Registration API, and in particular Fixtures, beg users to write static initialization bugs.
The BENCHMARK
registration macros execute during program startup, ie static initialization. Since the order of static initialization is unspecified, users who execute arbitrary code while registering benchmarks may encounter the static initialization order fiasco. Two common way users execute arbitrary code when registering benchmarks are (A) using the Benchmark::Apply
method, and (B) in the constructor of user-created Fixtures. The latter case causes problems a lot more frequently than the former.
Although users are partly to blame, especially in the fixture case, for even attempting to execute code during static initialization; it is still the libraries fault for providing API's which encourage users to do so.
We should attempt to come up with a fully functional API to register benchmarks at runtime, after static initialization is complete. And then we should re-write the documentation to encourage users to prefer the new interface over the old.
However... I still think the old interface is very useful, especially to those who are aware of it's dark corners. So I'm not suggesting removing it; but it would be nice if we were no longer responsible for causing initialization order bugs in users code.
Any and all ideas are welcome on how to improve the status quo. @dominichamon @pleroy Any ideas?
What is missing from our current runtime API?