@rlamy I'm opening this MR so that we can review/discuss it. First, I ran some benchmarks, comparing 3bf99c09018b (py3.7 branch) and eb7f65b143db (hpy-refactor-exceptions branch).
-
ujson: the branch is ~4% slower
-
piconumpy: the benchmark is not very stable, the branch is between 0% and 5% slower
-
microbenchmarks: we don't have an automated way to compare branches. I ran microbenchmarks three times for each branch and averaged the results, sorted from the biggest to the smallest slowdown. Many microbenchmarks are around ~10% slower.
... | py3.7 | branch | 3.7/branch |
---|---|---|---|
TestModule::test_noargs | 147,03 | 166,17 | 1,13 |
TestHeapType::test_getitem | 148,50 | 164,55 | 1,11 |
TestHeapType::test_noargs | 152,03 | 167,72 | 1,10 |
TestModule::test_allocate_int | 153,45 | 168,41 | 1,10 |
TestModule::test_call_with_tuple_and_dict | 2.993,40 | 3.265,25 | 1,09 |
TestModule::test_onearg_None | 193,08 | 209,90 | 1,09 |
TestType::test_getitem | 151,85 | 164,90 | 1,09 |
TestModule::test_onearg_int | 198,42 | 214,77 | 1,08 |
TestHeapType::test_onearg_None | 196,96 | 212,85 | 1,08 |
TestType::test_onearg_None | 199,11 | 212,51 | 1,07 |
TestModule::test_varargs | 441,83 | 465,85 | 1,05 |
TestHeapType::test_onearg_int | 206,94 | 218,02 | 1,05 |
TestType::test_noargs | 159,28 | 167,49 | 1,05 |
TestType::test_onearg_int | 208,48 | 217,80 | 1,04 |
TestHeapType::test_varargs | 447,01 | 461,53 | 1,03 |
TestType::test_varargs | 447,67 | 461,86 | 1,03 |
TestType::test_len | 90,98 | 91,79 | 1,01 |
TestType::test_method_lookup | 12,34 | 12,34 | 1,00 |
TestModule::test_call_with_tuple | 2.669,46 | 2.655,42 | 0,99 |
TestHeapType::test_method_lookup | 12,40 | 12,29 | 0,99 |
TestHeapType::test_len | 94,63 | 91,90 | 0,97 |
All in all, I am a bit worried that this approach might be suboptimal: if there are so many microbenchmarks 10% slower, maybe there will also be real-life code which will mostly use the costly features and show bigger slowdowns than ujson and piconumpy?
I think that the major point of this branch are:
- it makes it easier to make hpy and cpyext to cooperate
- it feels "less of a hack"
I don't think that point (2) is worth a potential 10% slowdown, but point (1) is important, of course. So my question is: what are the alternatives?
--HG-- branch : hpy-refactor-exceptions