divan icon indicating copy to clipboard operation
divan copied to clipboard

Any good way to set group option "on this level"?

Open antifuchs opened this issue 2 years ago • 4 comments

I'd love to separate out my project's benchmark definitions into multiple files, which divan seems to support via the [[bench]] cargo list method. However, I'd also like to set benchmark group options on each of these files, so that the #[bench] annotations don't have to repeat the same arguments over and over. Is there a good way to do that without introducing an "interim" module in each file?

I've tried:

Having a "main" file that uses submodules:

#[bench_group(threads=THREADS)
mod multi_threaded;

fn main() {
    divan::main();
}

That fails because procmacro annotations on external files are not stable.

Using a module-global bench_group annotation

Same as above, one main.rs file that uses mod multi_threaded; but inside the multi_threaded.rs file,

#![bench_group(threads=THREADS)

This fails because module-wide procmacro annotations aren't stable.

Using an interim module in each benchmark file

Here, put the main function into multi_threaded.rs, then pull in a submodule inside the file as a layer to set those benchmark options on:

fn main() {
    divan::main();
}

#[divan::bench_group(
    threads = THREADS,
)]
mod multi_threaded {
    // ...
}

This works, but results in a tree like:

     Running benches/multi_threaded.rs (target/release/deps/multi_threaded-77aac12e33714668)
multi_threaded                                                                                                                          fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ multi_threaded                                                                                                                                     │               │               │               │         │
   ├─ bench_direct                                                                                                                                    │               │               │               │         

...and I'd love to get rid of that interstitial module layer.

Possible solutions

  • Have a main function that accepts benchmark group options?
  • Allow setting the "default" benchmark options as globally-mutable state? (yikes?!)

antifuchs avatar Oct 07 '23 14:10 antifuchs

Would something like this work well for you?

/// Sets default options for all benchmarks and groups in the current module.
#[divan::bench_options]
fn divan_options(options: &mut divan::BenchOptions) {
    options.threads([1, 2, 4]).min_time(3).max_time(10);
}

0.2 will introduce a BenchOptions type. I'm open to adding this feature afterwards.

nvzqz avatar Jan 26 '25 19:01 nvzqz

Ooh, I like that! If options set the layer above could propagate down, that'd be the best!

antifuchs avatar Jan 26 '25 19:01 antifuchs

Yeah they would be inherited like the current #[divan::bench_group] behavior.

nvzqz avatar Jan 26 '25 19:01 nvzqz

That sounds perfect, I'd love that.

antifuchs avatar Jan 26 '25 19:01 antifuchs