Add configuration options to override Benchee default values

[PragTob]

Benchee is getting a lot of configuration options - which is great. But the more configurations options and formatters there are, the more people would probably prefer to configure some of them globally for their project.

Something like:

config :benchee, :options, %{} # fancy map or keyword list overriding default options

These options should then represent the new default options. So merge order would be something like: default_config <- app_config <- benchmark_config (<- meaning right overrides left)

[h4cc]

Default config can be found here: https://github.com/PragTob/benchee/blob/master/lib/benchee/config.ex#L156

Example for mix config: https://hexdocs.pm/mix/Mix.Config.html

[h4cc]

Is it ok for config to be included at compile time?

  @default_config %{
    parallel:   Application.fetch_env!(:benchee, :parallel, 1),
  }

[h4cc]

Working on this.

[devonestes]

So, in light of the recent configuration guidelines, should we still do this? I think we can totally get away without application config.

[PragTob]

It's something that I still don't fully understand.

I still think saying that I always want to use the HTML formatter might be useful and I dunno how else to best do it otherwise.

On the other hand, if people want that they could write their own little wrapper.

[devonestes]

We could do what Credo does and have a .benchee.exs file that's read as default configuration at runtime. That way we're not dealing with compile time configuration (which I believe is what those configuration guidelines are helping to prevent).

[PragTob]

Good point :clap:

Although until we have a way to specify paths without mentioning the file name this isn't essential to me. So I'll also remove the 1.0 milestone

[PragTob]

Thought about this one again: https://elixirforum.com/t/feedback-app-config-for-libraries-anti-pattern/60540/4?u=pragtob