hpy
hpy copied to clipboard
hpyproject
Microbench purepy and pypy profiling with VMProf
This gives:
$ make print_pypy
==================================== PyPy ======================================
cpy hpy purepy
---------------- ------------------- -------------------
TestModule::test_noargs 234.13 us 221.13 us [0.94] 96.82 us [0.41]
TestModule::test_onearg_None 264.33 us 248.83 us [0.94] 96.82 us [0.37]
TestModule::test_onearg_int 1693.34 us 235.71 us [0.14] 97.77 us [0.06]
TestModule::test_varargs 441.43 us 595.28 us [1.35] 97.42 us [0.22]
TestModule::test_call_with_tuple 4327.97 us 2488.76 us [0.58] 109.68 us [0.03]
TestModule::test_call_with_tuple_and_dict 10213.40 us 3148.42 us [0.31] 114.43 us [0.01]
TestModule::test_allocate_int 1762.43 us 357.62 us [0.20] 94.73 us [0.05]
TestModule::test_allocate_tuple 7789.97 us 6772.81 us [0.87] 302.46 us [0.04]
TestType::test_allocate_and_collect 12380.22 us 4259.69 us [0.34] 1295.53 us [0.10]
TestType::test_allocate_obj 7655.02 us 2815.65 us [0.37] 415.34 us [0.05]
TestType::test_method_lookup 11.43 us 45.19 us [3.95] 53.07 us [4.64]
TestType::test_noargs 220.70 us 274.12 us [1.24] 67.94 us [0.31]
TestType::test_onearg_None 227.91 us 309.57 us [1.36] 68.13 us [0.30]
TestType::test_onearg_int 1547.25 us 328.41 us [0.21] 65.97 us [0.04]
TestType::test_varargs 427.65 us 733.76 us [1.72] 63.23 us [0.15]
TestType::test_len 102.30 us 179.64 us [1.76] 74.09 us [0.72]
TestType::test_getitem 188.95 us 275.16 us [1.46] 68.65 us [0.36]
TestHeapType::test_allocate_and_collect 1255.27 us 4097.11 us [3.26] 1298.51 us [1.03]
TestHeapType::test_allocate_obj_and_die 2235.40 us 3150.59 us [1.41] 894.99 us [0.40]
TestHeapType::test_method_lookup 11.25 us 50.63 us [4.50] 57.72 us [5.13]
TestHeapType::test_noargs 236.17 us 272.70 us [1.15] 38.21 us [0.16]
TestHeapType::test_onearg_None 266.71 us 324.36 us [1.22] 40.45 us [0.15]
TestHeapType::test_onearg_int 1582.13 us 338.52 us [0.21] 46.12 us [0.03]
TestHeapType::test_varargs 442.07 us 713.21 us [1.61] 41.11 us [0.09]
TestHeapType::test_len 108.40 us 179.14 us [1.65] 38.31 us [0.35]
TestHeapType::test_getitem 217.56 us 274.32 us [1.26] 45.80 us [0.21]
================== 78 passed, 1 warning in 110.72s (0:01:50) ===================
and
$ make print_pypy_vs_cpy
=============== PyPy / CPy native (time ratio, smaller is better) ===============
cpyext HPy univ
TestModule::test_noargs 0.75 0.71
TestModule::test_onearg_None 0.97 0.92
TestModule::test_onearg_int 6.35 0.88
TestModule::test_varargs 0.87 1.17
TestModule::test_call_with_tuple 4.32 2.48
TestModule::test_call_with_tuple_and_dict 5.42 1.67
TestModule::test_allocate_int 4.12 0.84
TestModule::test_allocate_tuple 9.25 8.04
TestType::test_allocate_and_collect 19.48 6.70
TestType::test_allocate_obj 13.71 5.04
TestType::test_method_lookup 0.02 0.09
TestType::test_noargs 0.92 1.14
TestType::test_onearg_None 0.90 1.23
TestType::test_onearg_int 6.13 1.30
TestType::test_varargs 0.89 1.53
TestType::test_len 0.37 0.66
TestType::test_getitem 0.70 1.02
TestHeapType::test_allocate_and_collect 1.82 5.93
TestHeapType::test_allocate_obj_and_die 3.26 4.59
TestHeapType::test_method_lookup 0.02 0.10
TestHeapType::test_noargs 0.98 1.13
TestHeapType::test_onearg_None 1.03 1.25
TestHeapType::test_onearg_int 6.21 1.33
TestHeapType::test_varargs 0.92 1.48
TestHeapType::test_len 0.39 0.64
TestHeapType::test_getitem 0.78 0.98
Current outputs with d41024f:
$ make print_pypy
==================================== PyPy ======================================
cpy hpy purepy
---------------- ------------------- -------------------
TestModule::test_noargs 330.17 us 250.28 us [0.76] 88.54 us [0.27]
TestModule::test_onearg_None 360.27 us 289.94 us [0.80] 89.23 us [0.25]
TestModule::test_onearg_int 1841.98 us 302.51 us [0.16] 91.77 us [0.05]
TestModule::test_varargs 588.73 us 713.11 us [1.21] 85.92 us [0.15]
TestModule::test_call_with_tuple 6056.16 us 3408.84 us [0.56] 96.90 us [0.02]
TestModule::test_call_with_tuple_and_dict 14378.80 us 4306.48 us [0.30] 117.73 us [0.01]
TestModule::test_allocate_int 2134.12 us 483.09 us [0.23] 98.47 us [0.05]
TestModule::test_allocate_tuple 8973.39 us 8926.92 us [0.99] 89.90 us [0.01]
TestType::test_allocate_and_collect 14167.67 us 5576.36 us [0.39] 1728.27 us [0.12]
TestType::test_allocate_obj 9890.33 us 3591.33 us [0.36] 594.20 us [0.06]
TestType::test_allocate_obj_and_die 8520.09 us 4232.06 us [0.50] 1206.07 us [0.14]
TestType::test_method_lookup 23.81 us 43.35 us [1.82] 45.46 us [1.91]
TestType::test_noargs 295.91 us 332.50 us [1.12] 69.17 us [0.23]
TestType::test_onearg_None 345.05 us 407.50 us [1.18] 69.35 us [0.20]
TestType::test_onearg_int 1649.43 us 410.05 us [0.25] 69.38 us [0.04]
TestType::test_varargs 595.39 us 919.04 us [1.54] 66.07 us [0.11]
TestType::test_len 127.43 us 208.07 us [1.63] 76.61 us [0.60]
TestType::test_getitem 250.95 us 329.09 us [1.31] 69.46 us [0.28]
TestHeapType::test_allocate_and_collect 1729.86 us 5232.74 us [3.02] 1706.34 us [0.99]
TestHeapType::test_allocate_obj 563.75 us 3534.84 us [6.27] 552.34 us [0.98]
TestHeapType::test_allocate_obj_and_die 2921.14 us 4143.87 us [1.42] 1210.16 us [0.41]
TestHeapType::test_method_lookup 69.48 us 43.38 us [0.62] 45.10 us [0.65]
TestHeapType::test_noargs 457.18 us 285.74 us [0.63] 72.28 us [0.16]
TestHeapType::test_onearg_None 441.18 us 352.76 us [0.80] 72.22 us [0.16]
TestHeapType::test_onearg_int 1774.69 us 364.08 us [0.21] 72.34 us [0.04]
TestHeapType::test_varargs 687.50 us 881.91 us [1.28] 69.06 us [0.10]
TestHeapType::test_len 221.66 us 168.72 us [0.76] 76.23 us [0.34]
TestHeapType::test_getitem 330.49 us 291.42 us [0.88] 68.70 us [0.21]
and
$ make print_pypy_vs_cpy
=============== PyPy / CPy native (time ratio, smaller is better) ===============
cpyext HPy univ
TestModule::test_noargs 0.66 0.50
TestModule::test_onearg_None 0.80 0.64
TestModule::test_onearg_int 4.09 0.67
TestModule::test_varargs 0.75 0.91
TestModule::test_call_with_tuple 3.96 2.23
TestModule::test_call_with_tuple_and_dict 4.81 1.44
TestModule::test_allocate_int 3.02 0.68
TestModule::test_allocate_tuple 6.83 6.80
TestType::test_allocate_and_collect 14.09 5.54
TestType::test_allocate_obj 12.12 4.40
TestType::test_allocate_obj_and_die 9.65 4.79
TestType::test_method_lookup 0.03 0.05
TestType::test_noargs 0.75 0.84
TestType::test_onearg_None 0.78 0.93
TestType::test_onearg_int 3.90 0.97
TestType::test_varargs 0.80 1.23
TestType::test_len 0.30 0.49
TestType::test_getitem 0.59 0.78
TestHeapType::test_allocate_and_collect 1.69 5.12
TestHeapType::test_allocate_obj 0.65 4.07
TestHeapType::test_allocate_obj_and_die 2.93 4.16
TestHeapType::test_method_lookup 0.08 0.05
TestHeapType::test_noargs 1.18 0.74
TestHeapType::test_onearg_None 1.06 0.85
TestHeapType::test_onearg_int 4.22 0.87
TestHeapType::test_varargs 0.92 1.17
TestHeapType::test_len 0.53 0.40
TestHeapType::test_getitem 0.79 0.70
A first important task is to make PyPy HPy types as fast as PyPy cpyext heap types, which seems achievable (since HPy only has heap types).
Note: the differences between HPy Type and HPy HeapType should be related to instabilities of the measurements (since exactly the same code is used).