asv
asv copied to clipboard
Published
20 hours ago •
airspeed-velocity
airspeed-velocity
compare - group titles in --split mode are not present if --only-changed
Without --only-changed, there are "group titles" such as "Benchmarks that ..."
$> asv compare --machine hopa -s 0.11.x master
Benchmarks that have improved:
before after ratio
[0fc39f9f] [80807f23]
<0.11.8^0> <master>
failed 1.35±0.07s n/a api.supers.time_status [hopa/virtualenv-py3.7]
failed 2.85±0.3s n/a api.supers.time_status_recursive [hopa/virtualenv-py3.7]
- 3.88 1.42 0.37 core.runner.track_overhead_100ms [hopa/virtualenv-py3.7]
- -1.53 -3.03 1.98 core.runner.track_overhead_heavyout [hopa/virtualenv-py3.7]
- 24.15 15.42 0.64 core.runner.track_overhead_heavyout_online_process [hopa/virtualenv-py3.7]
failed 17.5±2ms n/a repo.gitrepo.time_get_content_info [hopa/virtualenv-py3.7]
failed 7.08±2ms n/a support.path.get_parent_paths.time_allsubmods_toplevel [hopa/virtualenv-py3.7]
failed 8.20±3ms n/a support.path.get_parent_paths.time_allsubmods_toplevel_only [hopa/virtualenv-py3.7]
failed 478±20ns n/a support.path.get_parent_paths.time_no_submods [hopa/virtualenv-py3.7]
failed 5.42±2ms n/a support.path.get_parent_paths.time_one_submod_subdir [hopa/virtualenv-py3.7]
failed 5.78±1ms n/a support.path.get_parent_paths.time_one_submod_toplevel [hopa/virtualenv-py3.7]
Benchmarks that have stayed the same:
before after ratio
[0fc39f9f] [80807f23]
<0.11.8^0> <master>
n/a 2.04±0.3s n/a api.supers.time_createadd [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
1.90±0.03s 1.99±0.2s 1.05 api.supers.time_createadd [hopa/virtualenv-py3.7]
n/a 3.11±0.04s n/a api.supers.time_createadd_to_dataset [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 11.7±0.07s n/a api.supers.time_installr [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
4.91±0.03s 13.9±0.3s ~2.83 api.supers.time_installr [hopa/virtualenv-py3.7]
n/a 408±8ms n/a api.supers.time_ls [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 3.61±0.03s n/a api.supers.time_ls_recursive [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
1.63±0.02s 5.57±0.4s ~3.43 api.supers.time_ls_recursive [hopa/virtualenv-py3.7]
n/a 3.62±0.1s n/a api.supers.time_ls_recursive_long_all [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
1.83±0.03s 4.26±0.7s ~2.32 api.supers.time_ls_recursive_long_all [hopa/virtualenv-py3.7]
failed failed n/a api.supers.time_rev_createadd [hopa/virtualenv-py2.7]
failed failed n/a api.supers.time_rev_createadd [hopa/virtualenv-py3.7]
failed failed n/a api.supers.time_rev_createadd_to_dataset [hopa/virtualenv-py2.7]
failed failed n/a api.supers.time_rev_createadd_to_dataset [hopa/virtualenv-py3.7]
n/a 1.30±0.1s n/a api.supers.time_status [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
failed failed n/a api.supers.time_status [hopa/virtualenv-py2.7]
n/a 2.49±2s n/a api.supers.time_status_recursive [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
failed failed n/a api.supers.time_status_recursive [hopa/virtualenv-py2.7]
n/a 282±9ms n/a api.supers.time_subdatasets [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 927±30ms n/a api.supers.time_subdatasets_recursive [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 228±10ms n/a api.supers.time_subdatasets_recursive_first [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 9.42±0.2s n/a api.supers.time_uninstall [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
2.73±0.05s 9.41±0.2s ~3.45 api.supers.time_uninstall [hopa/virtualenv-py3.7]
n/a 944±6ms n/a api.testds.time_create_test_dataset1 [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 5.71±0.1s n/a api.testds.time_create_test_dataset2x2 [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
4.22±0.05s 6.11±0.02s ~1.45 api.testds.time_create_test_dataset2x2 [hopa/virtualenv-py3.7]
n/a 66.6±9ns n/a check_asv.Benchmarks.time_upper [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
61.7±0.8ns 76.2±20ns ~1.23 check_asv.Benchmarks.time_upper [hopa/virtualenv-py3.7]
n/a 66.5±10ns n/a check_asv.Suite1.time_upper [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
60.8±0.8ns 62.5±4ns 1.03 check_asv.Suite1.time_upper [hopa/virtualenv-py3.7]
n/a 63.0±10ns n/a check_asv.Suite2.time_upper [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
60.5±0.5ns 91.4±30ns ~1.51 check_asv.Suite2.time_upper [hopa/virtualenv-py3.7]
n/a 2.35±0.1ms n/a core.runner.time_echo [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 2.72±0.4ms n/a core.runner.time_echo_gitrunner [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
2.24±0.05ms 2.78±0.3ms ~1.24 core.runner.time_echo_gitrunner [hopa/virtualenv-py3.7]
n/a 3.61 n/a core.runner.track_overhead_100ms [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 316.78 n/a core.runner.track_overhead_echo [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 3.44 n/a core.runner.track_overhead_heavyout [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 24.95 n/a core.runner.track_overhead_heavyout_online_process [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 10.62 n/a core.runner.track_overhead_heavyout_online_through [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 1.09±0.2s n/a core.startup.time_help_np [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 172±20ms n/a core.startup.time_import [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
109±2ms 135±4ms ~1.24 core.startup.time_import [hopa/virtualenv-py3.7]
n/a 982±200ms n/a core.startup.time_import_api [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
n/a 19.2±3ms n/a repo.gitrepo.time_get_content_info [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
failed failed n/a repo.gitrepo.time_get_content_info [hopa/virtualenv-py2.7]
n/a 6.15±0.3ms n/a support.path.get_parent_paths.time_allsubmods_toplevel [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
failed failed n/a support.path.get_parent_paths.time_allsubmods_toplevel [hopa/virtualenv-py2.7]
n/a 6.33±1ms n/a support.path.get_parent_paths.time_allsubmods_toplevel_only [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
failed failed n/a support.path.get_parent_paths.time_allsubmods_toplevel_only [hopa/virtualenv-py2.7]
n/a 258±10ns n/a support.path.get_parent_paths.time_no_submods [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
failed failed n/a support.path.get_parent_paths.time_no_submods [hopa/virtualenv-py2.7]
n/a 4.77±0.2ms n/a support.path.get_parent_paths.time_one_submod_subdir [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
failed failed n/a support.path.get_parent_paths.time_one_submod_subdir [hopa/virtualenv-py2.7]
n/a 5.90±0.9ms n/a support.path.get_parent_paths.time_one_submod_toplevel [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
failed failed n/a support.path.get_parent_paths.time_one_submod_toplevel [hopa/virtualenv-py2.7]
n/a 1.06±0.2m n/a usecases.study_forrest.time_make_studyforrest_mockup [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
52.1±0.01s 59.2±1s ~1.14 usecases.study_forrest.time_make_studyforrest_mockup [hopa/virtualenv-py3.7]
Benchmarks that have got worse:
before after ratio
[0fc39f9f] [80807f23]
<0.11.8^0> <master>
! 2.16±0.09s failed n/a api.supers.time_createadd [hopa/virtualenv-py2.7]
! 2.13±0.2s failed n/a api.supers.time_createadd_to_dataset [hopa/virtualenv-py2.7]
+ 1.96±0.04s 3.11±0.3s 1.59 api.supers.time_createadd_to_dataset [hopa/virtualenv-py3.7]
! n/a failed n/a api.supers.time_diff [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
! 279±50ms failed n/a api.supers.time_diff [hopa/virtualenv-py2.7]
! 183±9ms failed n/a api.supers.time_diff [hopa/virtualenv-py3.7]
! n/a failed n/a api.supers.time_diff_recursive [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
! 463±40ms failed n/a api.supers.time_diff_recursive [hopa/virtualenv-py2.7]
! 446±10ms failed n/a api.supers.time_diff_recursive [hopa/virtualenv-py3.7]
! 5.68±0.1s failed n/a api.supers.time_installr [hopa/virtualenv-py2.7]
! 291±5ms failed n/a api.supers.time_ls [hopa/virtualenv-py2.7]
+ 248±4ms 507±20ms 2.04 api.supers.time_ls [hopa/virtualenv-py3.7]
! 1.57±0.04s failed n/a api.supers.time_ls_recursive [hopa/virtualenv-py2.7]
! 1.92±0.05s failed n/a api.supers.time_ls_recursive_long_all [hopa/virtualenv-py2.7]
! n/a failed n/a api.supers.time_remove [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
! 2.80±0.07s failed n/a api.supers.time_remove [hopa/virtualenv-py2.7]
! 2.61±0.06s failed n/a api.supers.time_remove [hopa/virtualenv-py3.7]
! n/a failed n/a api.supers.time_rev_createadd [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
! n/a failed n/a api.supers.time_rev_createadd_to_dataset [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
! 59.1±10ms failed n/a api.supers.time_subdatasets [hopa/virtualenv-py2.7]
+ 51.7±3ms 501±30ms 9.68 api.supers.time_subdatasets [hopa/virtualenv-py3.7]
! 129±30ms failed n/a api.supers.time_subdatasets_recursive [hopa/virtualenv-py2.7]
+ 148±6ms 1.03±0.07s 6.96 api.supers.time_subdatasets_recursive [hopa/virtualenv-py3.7]
! 74.0±8ms failed n/a api.supers.time_subdatasets_recursive_first [hopa/virtualenv-py2.7]
+ 51.2±3ms 254±30ms 4.95 api.supers.time_subdatasets_recursive_first [hopa/virtualenv-py3.7]
! 2.87±0.08s failed n/a api.supers.time_uninstall [hopa/virtualenv-py2.7]
! 546±30ms failed n/a api.testds.time_create_test_dataset1 [hopa/virtualenv-py2.7]
+ 538±5ms 1.01±0.07s 1.87 api.testds.time_create_test_dataset1 [hopa/virtualenv-py3.7]
! 3.25±0.1s failed n/a api.testds.time_create_test_dataset2x2 [hopa/virtualenv-py2.7]
! 70.1±4ns failed n/a check_asv.Benchmarks.time_upper [hopa/virtualenv-py2.7]
! 69.6±1ns failed n/a check_asv.Suite1.time_upper [hopa/virtualenv-py2.7]
! 67.0±0.5ns failed n/a check_asv.Suite2.time_upper [hopa/virtualenv-py2.7]
! 2.80±0.2ms failed n/a core.runner.time_echo [hopa/virtualenv-py2.7]
+ 1.98±0.04ms 2.28±0.04ms 1.15 core.runner.time_echo [hopa/virtualenv-py3.7]
! 3.31±0.2ms failed n/a core.runner.time_echo_gitrunner [hopa/virtualenv-py2.7]
! 4.07 failed n/a core.runner.track_overhead_100ms [hopa/virtualenv-py2.7]
! 238.02 failed n/a core.runner.track_overhead_echo [hopa/virtualenv-py2.7]
+ 214.66 279.46 1.30 core.runner.track_overhead_echo [hopa/virtualenv-py3.7]
! -1.81 failed n/a core.runner.track_overhead_heavyout [hopa/virtualenv-py2.7]
! 400.58 failed n/a core.runner.track_overhead_heavyout_online_process [hopa/virtualenv-py2.7]
! 373.56 failed n/a core.runner.track_overhead_heavyout_online_through [hopa/virtualenv-py2.7]
+ 24.97 62.67 2.51 core.runner.track_overhead_heavyout_online_through [hopa/virtualenv-py3.7]
! 393±7ms failed n/a core.startup.time_help_np [hopa/virtualenv-py2.7]
+ 434±4ms 631±10ms 1.45 core.startup.time_help_np [hopa/virtualenv-py3.7]
! 82.8±3ms failed n/a core.startup.time_import [hopa/virtualenv-py2.7]
! 363±20ms failed n/a core.startup.time_import_api [hopa/virtualenv-py2.7]
+ 379±2ms 508±50ms 1.34 core.startup.time_import_api [hopa/virtualenv-py3.7]
! 54.3±0.04s failed n/a usecases.study_forrest.time_make_studyforrest_mockup [hopa/virtualenv-py2.7]
and those are not printed at all (e.g. to describe what got better or worse) if --only-changed
$> asv compare --machine hopa --only-changed -s 0.11.x master
before after ratio
[0fc39f9f] [80807f23]
<0.11.8^0> <master>
- 3.88 1.42 0.37 core.runner.track_overhead_100ms [hopa/virtualenv-py3.7]
- -1.53 -3.03 1.98 core.runner.track_overhead_heavyout [hopa/virtualenv-py3.7]
- 24.15 15.42 0.64 core.runner.track_overhead_heavyout_online_process [hopa/virtualenv-py3.7]
before after ratio
[0fc39f9f] [80807f23]
<0.11.8^0> <master>
! 2.16±0.09s failed n/a api.supers.time_createadd [hopa/virtualenv-py2.7]
! 2.13±0.2s failed n/a api.supers.time_createadd_to_dataset [hopa/virtualenv-py2.7]
+ 1.96±0.04s 3.11±0.3s 1.59 api.supers.time_createadd_to_dataset [hopa/virtualenv-py3.7]
! n/a failed n/a api.supers.time_diff [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
! 279±50ms failed n/a api.supers.time_diff [hopa/virtualenv-py2.7]
! 183±9ms failed n/a api.supers.time_diff [hopa/virtualenv-py3.7]
! n/a failed n/a api.supers.time_diff_recursive [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
! 463±40ms failed n/a api.supers.time_diff_recursive [hopa/virtualenv-py2.7]
! 446±10ms failed n/a api.supers.time_diff_recursive [hopa/virtualenv-py3.7]
! 5.68±0.1s failed n/a api.supers.time_installr [hopa/virtualenv-py2.7]
! 291±5ms failed n/a api.supers.time_ls [hopa/virtualenv-py2.7]
+ 248±4ms 507±20ms 2.04 api.supers.time_ls [hopa/virtualenv-py3.7]
! 1.57±0.04s failed n/a api.supers.time_ls_recursive [hopa/virtualenv-py2.7]
! 1.92±0.05s failed n/a api.supers.time_ls_recursive_long_all [hopa/virtualenv-py2.7]
! n/a failed n/a api.supers.time_remove [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
! 2.80±0.07s failed n/a api.supers.time_remove [hopa/virtualenv-py2.7]
! 2.61±0.06s failed n/a api.supers.time_remove [hopa/virtualenv-py3.7]
! n/a failed n/a api.supers.time_rev_createadd [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
! n/a failed n/a api.supers.time_rev_createadd_to_dataset [hopa/existing-py_home_yoh_proj_datalad_datalad-master_venvs_dev3_bin_python3]
! 59.1±10ms failed n/a api.supers.time_subdatasets [hopa/virtualenv-py2.7]
+ 51.7±3ms 501±30ms 9.68 api.supers.time_subdatasets [hopa/virtualenv-py3.7]
! 129±30ms failed n/a api.supers.time_subdatasets_recursive [hopa/virtualenv-py2.7]
+ 148±6ms 1.03±0.07s 6.96 api.supers.time_subdatasets_recursive [hopa/virtualenv-py3.7]
! 74.0±8ms failed n/a api.supers.time_subdatasets_recursive_first [hopa/virtualenv-py2.7]
+ 51.2±3ms 254±30ms 4.95 api.supers.time_subdatasets_recursive_first [hopa/virtualenv-py3.7]
! 2.87±0.08s failed n/a api.supers.time_uninstall [hopa/virtualenv-py2.7]
! 546±30ms failed n/a api.testds.time_create_test_dataset1 [hopa/virtualenv-py2.7]
+ 538±5ms 1.01±0.07s 1.87 api.testds.time_create_test_dataset1 [hopa/virtualenv-py3.7]
! 3.25±0.1s failed n/a api.testds.time_create_test_dataset2x2 [hopa/virtualenv-py2.7]
! 70.1±4ns failed n/a check_asv.Benchmarks.time_upper [hopa/virtualenv-py2.7]
! 69.6±1ns failed n/a check_asv.Suite1.time_upper [hopa/virtualenv-py2.7]
! 67.0±0.5ns failed n/a check_asv.Suite2.time_upper [hopa/virtualenv-py2.7]
! 2.80±0.2ms failed n/a core.runner.time_echo [hopa/virtualenv-py2.7]
+ 1.98±0.04ms 2.28±0.04ms 1.15 core.runner.time_echo [hopa/virtualenv-py3.7]
! 3.31±0.2ms failed n/a core.runner.time_echo_gitrunner [hopa/virtualenv-py2.7]
! 4.07 failed n/a core.runner.track_overhead_100ms [hopa/virtualenv-py2.7]
! 238.02 failed n/a core.runner.track_overhead_echo [hopa/virtualenv-py2.7]
+ 214.66 279.46 1.30 core.runner.track_overhead_echo [hopa/virtualenv-py3.7]
! -1.81 failed n/a core.runner.track_overhead_heavyout [hopa/virtualenv-py2.7]
! 400.58 failed n/a core.runner.track_overhead_heavyout_online_process [hopa/virtualenv-py2.7]
! 373.56 failed n/a core.runner.track_overhead_heavyout_online_through [hopa/virtualenv-py2.7]
+ 24.97 62.67 2.51 core.runner.track_overhead_heavyout_online_through [hopa/virtualenv-py3.7]
! 393±7ms failed n/a core.startup.time_help_np [hopa/virtualenv-py2.7]
+ 434±4ms 631±10ms 1.45 core.startup.time_help_np [hopa/virtualenv-py3.7]
! 82.8±3ms failed n/a core.startup.time_import [hopa/virtualenv-py2.7]
! 363±20ms failed n/a core.startup.time_import_api [hopa/virtualenv-py2.7]
+ 379±2ms 508±50ms 1.34 core.startup.time_import_api [hopa/virtualenv-py3.7]
! 54.3±0.04s failed n/a usecases.study_forrest.time_make_studyforrest_mockup [hopa/virtualenv-py2.7]
sure thing they could be deduced, but the same is true for full (without --only-changed) mode, so might be better to be consistent
