Joshua Bronson

Thanks so much for those examples! Look forward to taking a closer look soon.

Wow, thanks for taking a closer look, this looks super helpful! I'm about to get pulled away to a conference for the next 10 days, but psyched to dig into...

Hi @ionelmc, I took a closer look and this was definitely helpful, thank you. Several improvements based on your tips in my latest version: https://github.com/jab/bidict/blob/0.12.0-dev/tests/test_benchmark.py There are still some unanswered...

> ​You could only show the more interesting columns, eg: --benchmark-columns=min,stddev​ When I try that I get `py.test: error: unrecognized arguments: --benchmark-columns=min,stddev` and I don't see it documented at http://pytest-benchmark.readthedocs.org/en/stable/usage.html#commandline-options...

> Alternatively you can use the the group hook: http://pytest-benchmark.readthedocs.org/en/v3.0.0/hooks.html#pytest_benchmark.hookspec.pytest_benchmark_group_stats I wrote something based on the example in the docs but it's never getting called: ``` File "/Library/Python/2.7/site-packages/pytest_benchmark/plugin.py", line 986,...

In https://github.com/jab/bidict/blob/0.12.0-dev/tests/test_benchmark.py right after the imports.

In case you didn't see this other question in https://github.com/jab/bidict/blob/0.12.0-dev/tests/test_benchmark.py#L104, I'll paste here: ``` python # TODO: iterations=100 causes: ValueError: Can't use more than 1 `iterations` with a `setup` function....

Thanks @ionelmc, that did the trick. Benchmarks are now properly getting grouped by (test, input size): ``` ------------------------------------------------------------------------ benchmark 'test_get_key_by_val[11]': 2 tests ----------------------------------------------------------------------- Name (time in ns) Min Max Mean...

Thanks @ionelmc, I'd seen that and it's a nice explanation. I'm curious to understand it a bit deeper (e.g. _why_ some of my tests require only 1 iteration where others...

What would really be helpful is if there were a way to set up pytest-benchmark and Travis-CI so that: - the benchmark data for the latest build of master were...