Microbench purepy and pypy profiling with VMProf

[paugier]

This gives:

$ make print_pypy
==================================== PyPy ======================================
                                                     cpy                    hpy                 purepy
                                        ----------------    -------------------    -------------------
TestModule::test_noargs                        234.13 us       221.13 us [0.94]        96.82 us [0.41]
TestModule::test_onearg_None                   264.33 us       248.83 us [0.94]        96.82 us [0.37]
TestModule::test_onearg_int                   1693.34 us       235.71 us [0.14]        97.77 us [0.06]
TestModule::test_varargs                       441.43 us       595.28 us [1.35]        97.42 us [0.22]
TestModule::test_call_with_tuple              4327.97 us      2488.76 us [0.58]       109.68 us [0.03]
TestModule::test_call_with_tuple_and_dict     10213.40 us      3148.42 us [0.31]       114.43 us [0.01]
TestModule::test_allocate_int                 1762.43 us       357.62 us [0.20]        94.73 us [0.05]
TestModule::test_allocate_tuple               7789.97 us      6772.81 us [0.87]       302.46 us [0.04]
TestType::test_allocate_and_collect          12380.22 us      4259.69 us [0.34]      1295.53 us [0.10]
TestType::test_allocate_obj                   7655.02 us      2815.65 us [0.37]       415.34 us [0.05]
TestType::test_method_lookup                    11.43 us        45.19 us [3.95]        53.07 us [4.64]
TestType::test_noargs                          220.70 us       274.12 us [1.24]        67.94 us [0.31]
TestType::test_onearg_None                     227.91 us       309.57 us [1.36]        68.13 us [0.30]
TestType::test_onearg_int                     1547.25 us       328.41 us [0.21]        65.97 us [0.04]
TestType::test_varargs                         427.65 us       733.76 us [1.72]        63.23 us [0.15]
TestType::test_len                             102.30 us       179.64 us [1.76]        74.09 us [0.72]
TestType::test_getitem                         188.95 us       275.16 us [1.46]        68.65 us [0.36]
TestHeapType::test_allocate_and_collect       1255.27 us      4097.11 us [3.26]      1298.51 us [1.03]
TestHeapType::test_allocate_obj_and_die       2235.40 us      3150.59 us [1.41]       894.99 us [0.40]
TestHeapType::test_method_lookup                11.25 us        50.63 us [4.50]        57.72 us [5.13]
TestHeapType::test_noargs                      236.17 us       272.70 us [1.15]        38.21 us [0.16]
TestHeapType::test_onearg_None                 266.71 us       324.36 us [1.22]        40.45 us [0.15]
TestHeapType::test_onearg_int                 1582.13 us       338.52 us [0.21]        46.12 us [0.03]
TestHeapType::test_varargs                     442.07 us       713.21 us [1.61]        41.11 us [0.09]
TestHeapType::test_len                         108.40 us       179.14 us [1.65]        38.31 us [0.35]
TestHeapType::test_getitem                     217.56 us       274.32 us [1.26]        45.80 us [0.21]

================== 78 passed, 1 warning in 110.72s (0:01:50) ===================

and

$ make print_pypy_vs_cpy 
=============== PyPy / CPy native (time ratio, smaller is better) ===============
                                             cpyext    HPy univ
TestModule::test_noargs                        0.75       0.71
TestModule::test_onearg_None                   0.97       0.92
TestModule::test_onearg_int                    6.35       0.88
TestModule::test_varargs                       0.87       1.17
TestModule::test_call_with_tuple               4.32       2.48
TestModule::test_call_with_tuple_and_dict      5.42       1.67
TestModule::test_allocate_int                  4.12       0.84
TestModule::test_allocate_tuple                9.25       8.04
TestType::test_allocate_and_collect           19.48       6.70
TestType::test_allocate_obj                   13.71       5.04
TestType::test_method_lookup                   0.02       0.09
TestType::test_noargs                          0.92       1.14
TestType::test_onearg_None                     0.90       1.23
TestType::test_onearg_int                      6.13       1.30
TestType::test_varargs                         0.89       1.53
TestType::test_len                             0.37       0.66
TestType::test_getitem                         0.70       1.02
TestHeapType::test_allocate_and_collect        1.82       5.93
TestHeapType::test_allocate_obj_and_die        3.26       4.59
TestHeapType::test_method_lookup               0.02       0.10
TestHeapType::test_noargs                      0.98       1.13
TestHeapType::test_onearg_None                 1.03       1.25
TestHeapType::test_onearg_int                  6.21       1.33
TestHeapType::test_varargs                     0.92       1.48
TestHeapType::test_len                         0.39       0.64
TestHeapType::test_getitem                     0.78       0.98

[paugier]

Current outputs with d41024f:

$ make print_pypy
==================================== PyPy ======================================
                                                     cpy                    hpy                 purepy
                                        ----------------    -------------------    -------------------
TestModule::test_noargs                        330.17 us       250.28 us [0.76]        88.54 us [0.27]
TestModule::test_onearg_None                   360.27 us       289.94 us [0.80]        89.23 us [0.25]
TestModule::test_onearg_int                   1841.98 us       302.51 us [0.16]        91.77 us [0.05]
TestModule::test_varargs                       588.73 us       713.11 us [1.21]        85.92 us [0.15]
TestModule::test_call_with_tuple              6056.16 us      3408.84 us [0.56]        96.90 us [0.02]
TestModule::test_call_with_tuple_and_dict     14378.80 us      4306.48 us [0.30]       117.73 us [0.01]
TestModule::test_allocate_int                 2134.12 us       483.09 us [0.23]        98.47 us [0.05]
TestModule::test_allocate_tuple               8973.39 us      8926.92 us [0.99]        89.90 us [0.01]
TestType::test_allocate_and_collect          14167.67 us      5576.36 us [0.39]      1728.27 us [0.12]
TestType::test_allocate_obj                   9890.33 us      3591.33 us [0.36]       594.20 us [0.06]
TestType::test_allocate_obj_and_die           8520.09 us      4232.06 us [0.50]      1206.07 us [0.14]
TestType::test_method_lookup                    23.81 us        43.35 us [1.82]        45.46 us [1.91]
TestType::test_noargs                          295.91 us       332.50 us [1.12]        69.17 us [0.23]
TestType::test_onearg_None                     345.05 us       407.50 us [1.18]        69.35 us [0.20]
TestType::test_onearg_int                     1649.43 us       410.05 us [0.25]        69.38 us [0.04]
TestType::test_varargs                         595.39 us       919.04 us [1.54]        66.07 us [0.11]
TestType::test_len                             127.43 us       208.07 us [1.63]        76.61 us [0.60]
TestType::test_getitem                         250.95 us       329.09 us [1.31]        69.46 us [0.28]
TestHeapType::test_allocate_and_collect       1729.86 us      5232.74 us [3.02]      1706.34 us [0.99]
TestHeapType::test_allocate_obj                563.75 us      3534.84 us [6.27]       552.34 us [0.98]
TestHeapType::test_allocate_obj_and_die       2921.14 us      4143.87 us [1.42]      1210.16 us [0.41]
TestHeapType::test_method_lookup                69.48 us        43.38 us [0.62]        45.10 us [0.65]
TestHeapType::test_noargs                      457.18 us       285.74 us [0.63]        72.28 us [0.16]
TestHeapType::test_onearg_None                 441.18 us       352.76 us [0.80]        72.22 us [0.16]
TestHeapType::test_onearg_int                 1774.69 us       364.08 us [0.21]        72.34 us [0.04]
TestHeapType::test_varargs                     687.50 us       881.91 us [1.28]        69.06 us [0.10]
TestHeapType::test_len                         221.66 us       168.72 us [0.76]        76.23 us [0.34]
TestHeapType::test_getitem                     330.49 us       291.42 us [0.88]        68.70 us [0.21]

and

$ make print_pypy_vs_cpy 
=============== PyPy / CPy native (time ratio, smaller is better) ===============
                                             cpyext    HPy univ
TestModule::test_noargs                        0.66       0.50
TestModule::test_onearg_None                   0.80       0.64
TestModule::test_onearg_int                    4.09       0.67
TestModule::test_varargs                       0.75       0.91
TestModule::test_call_with_tuple               3.96       2.23
TestModule::test_call_with_tuple_and_dict      4.81       1.44
TestModule::test_allocate_int                  3.02       0.68
TestModule::test_allocate_tuple                6.83       6.80
TestType::test_allocate_and_collect           14.09       5.54
TestType::test_allocate_obj                   12.12       4.40
TestType::test_allocate_obj_and_die            9.65       4.79
TestType::test_method_lookup                   0.03       0.05
TestType::test_noargs                          0.75       0.84
TestType::test_onearg_None                     0.78       0.93
TestType::test_onearg_int                      3.90       0.97
TestType::test_varargs                         0.80       1.23
TestType::test_len                             0.30       0.49
TestType::test_getitem                         0.59       0.78
TestHeapType::test_allocate_and_collect        1.69       5.12
TestHeapType::test_allocate_obj                0.65       4.07
TestHeapType::test_allocate_obj_and_die        2.93       4.16
TestHeapType::test_method_lookup               0.08       0.05
TestHeapType::test_noargs                      1.18       0.74
TestHeapType::test_onearg_None                 1.06       0.85
TestHeapType::test_onearg_int                  4.22       0.87
TestHeapType::test_varargs                     0.92       1.17
TestHeapType::test_len                         0.53       0.40
TestHeapType::test_getitem                     0.79       0.70

A first important task is to make PyPy HPy types as fast as PyPy cpyext heap types, which seems achievable (since HPy only has heap types).

Note: the differences between HPy Type and HPy HeapType should be related to instabilities of the measurements (since exactly the same code is used).