[FR] Add possibility for grouping benchmarks and disabling some groups

[kononovk]

Hello! We use Google benchmark to compare our library with others. And we often need to group benchmarks into categories (e.g. Lib1, Lib2, Lib3). This is convenient to disable some libraries or get json something like:

{
 "Bench1": [{ Group1 results }, { Group2 results }, ... ]
}

It is also very convenient to run only some groups of benchmarks by the flag passed to cmake. Example: cmake -DGROUP1=ON, -DGROUP3=ON

Now we do it with special macros and EmptyBenchmark class: https://github.com/YACLib/Bench/blob/master/util/bench.hpp#L50

But this is not very convenient, and I would like some more universal interface. Are similar improvements planned in the future? Maybe you have some general advices on how to improve the organization of our benchmarks

[dmah42]

there's nothing in the plan for this, but i like your solution.

when the library was first conceived (quite a while ago) this similar thing was managed by naming benchmarks appropriately and using the benchmark_filter to include or exclude benchmarks. obviously this isn't particularly friendly or flexible.

is it important that the disabled benchmarks are removed from the executable, or just that they are a no-op/not run?

[LebedevRI]

I don't have any particular ideas how this could be achieved (is a single group enough, or will they be nested? why group and not tag? what about printing that info in console? ...), but whatever happens, the JSON should remain flat, not nested.

[kononovk]

there's nothing in the plan for this, but i like your solution.

when the library was first conceived (quite a while ago) this similar thing was managed by naming benchmarks appropriately and using the benchmark_filter to include or exclude benchmarks. obviously this isn't particularly friendly or flexible.

is it important that the disabled benchmarks are removed from the executable, or just that they are a no-op/not run?

I think "they are a no-op/not run" is enough