Allow reading config from tox.ini / setup.cfg?

[aldanor]

Seeing as there's a growing list of options that barely fits one help screen, maybe it would make sense to think about reading the default from config file same as most other tools do (tox itself, coverage, pytest, all sorts of linters)?

So instead of passing a bajillion of command line arguments, you could just add a section in your tox.ini like so (just making these up):

[benchmark]
columns = min, max
precision = auto
report = term, json
json = foo.json

[ionelmc]

Just for refference, the complete list of current options:

[benchmark]
autosave = false
calibration_precision = 10
compare =
compare_fail =
disable_gc = false
group_by = group
histogram =
json =
max_time = 1.0
min_rounds = 5
min_time = 0.000005
only = false
save =
save_data =
skip = false
sort = min
storage =
timer = time.perf_counter
verbose = false
warmup = false
warmup_iterations = 100000

And new options:

columns = min, max
precision = auto
report = term, json

[aldanor]

Whoah that's a lot of options, this definitely deserves a config file section!

[ionelmc]

Maybe we should use some namespaces, eg:

[benchmark:report]
sort = min
compare = 
compare_fail = 

[benchmark:report:term]
class = mod.MyCustomRenderer
precision = auto
columns  = min, max

[anarcat]

+1

[ionelmc]

Another idea is to use environment variables

[aldanor]

@ionelmc Haven't been using pytest-benchmark for quite a while, now the time has come again, I've searched for ways to configure it and... found my own ticket :)

Wondering if there are any news on this front, or help needed?

[ionelmc]

@aldanor ah sorry I haven't done anything on this yet. Do you want to help with a PR?

[aldanor]

Yet another thought... pyproject.toml? (seeing as more and more libraries and projects are moving towards it these days)

[ionelmc]

Worth checking out if tox/pytest will move to toml (there was some discussion about that but I forgot where) and just do what they do.

[matham]

I'm a little confused about whether there's currently some support for reading config from a file. E.g. https://github.com/ionelmc/pytest-benchmark/issues/27 and in the docs:

--benchmark-enable
   Forcibly enable benchmarks. Use this option to override –benchmark-disable (in case you have it in pytest configuration).

seem to indicate there's already a way to read from pytest configuration files since it's causally mentioned!?

[ionelmc]

Look no further: https://docs.pytest.org/en/stable/customize.html (eg: addopts = --benchmark-something). Tho I don't think it's a good idea to have --benchmark-enable in the config file - you'd never be able to disable the benchmarks.