pyperformance icon indicating copy to clipboard operation
pyperformance copied to clipboard

Add logging message to notify user of skipped benchmarks

Open bclodius opened this issue 2 years ago • 1 comments

Background

Hello!

I was recently running a number of benchmarks to analyze a migration to different python3 versions as well as different server infrastructure.

In my first 30-60 minutes playing with this library I was very confused that my results would only show benchmarks for comprehensions and sqlglot. After some breadcrumbs and print statements I finally came to realize that a large majority of the benchmarks require >=3.8. I was testing on a version of python 3.7.

I was hoping this small enhancement would help other users in future to prevent hair pulling and headaches!

Testing

Before

...
pyperformance run -b all
...
( 1/2) creating venv for benchmark (comprehensions)
...
( 2/2) creating venv for benchmark (sqlglot)
...

After

pyperformance run -b all
WARNING:root:Benchmark: 2to3 is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: async_generators is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: async_tree is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: async_tree_cpu_io_mixed is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: async_tree_io is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: async_tree_memoization is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_cpu_io_mixed is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_io is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_memoization is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_tg is skipped because python version required is >=3.11 but found 3.7.16
WARNING:root:Benchmark: async_tree_cpu_io_mixed_tg is skipped because python version required is >=3.11 but found 3.7.16
WARNING:root:Benchmark: async_tree_io_tg is skipped because python version required is >=3.11 but found 3.7.16
WARNING:root:Benchmark: async_tree_memoization_tg is skipped because python version required is >=3.11 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_tg is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_cpu_io_mixed_tg is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_io_tg is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_memoization_tg is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: asyncio_tcp is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: asyncio_tcp_ssl is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: asyncio_websockets is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: concurrent_imap is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: coroutines is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: coverage is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: gc_traversal is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: gc_collect is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: generators is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: chameleon is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: chaos is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: crypto_pyaes is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: dask is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: deepcopy is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: deltablue is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: django_template is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: dulwich_log is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: docutils is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: fannkuch is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: float is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: genshi is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: go is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: hexiom is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: html5lib is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: json_dumps is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: json_loads is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: logging is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: mako is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: mdp is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: meteor_contest is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: nbody is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: nqueens is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pathlib is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pickle is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pickle_dict is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pickle_list is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pickle_pure_python is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pidigits is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pprint is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pyflate is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: python_startup is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: python_startup_no_site is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: raytrace is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: regex_compile is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: regex_dna is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: regex_effbot is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: regex_v8 is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: richards is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: richards_super is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: scimark is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: spectral_norm is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: sqlalchemy_declarative is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: sqlalchemy_imperative is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: sqlite_synth is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: sympy is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: telco is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: tomli_loads is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: tornado_http is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: typing_runtime_protocols is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: unpack_sequence is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: unpickle is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: unpickle_list is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: unpickle_pure_python is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: xml_etree is skipped because python version required is >=3.8 but found 3.7.16

bclodius avatar Oct 09 '23 01:10 bclodius

@hugovk thanks for your feedback, I believe I've incorporated it all.

bclodius avatar Oct 09 '23 20:10 bclodius