PGO enabling

Merged Bitbucket Importer requested to merge bitbucket/merged-pr-540 into branch/default

Created originally on Bitbucket by Dodan (Dodan Mihai)

Was already merged in Bitbucket before import, marked as merged by the import user

Hello, I would like to propose a solution for the utilizing of profile guided optimizations(profopt) for PyPy(issues/2398)

Changes:

  • The --profopt option is now a boolean option instead of string and I eliminated the --no-profopt option altogether
  • Added the python's regression tests in pypy/goal/regrtest to be the training set for profile guided optimizations. The training workload is now static and the rule for profopt will need the regression tests in pypy/goal/regrtest
  • Removed all the profopt classes and profopt related functions in rpython/translator/c/genc.py, and replaced them with 2 rules and 1 definition for the Makefile.

The definition describes the profopt target of the Makefile. This was needed because PyPy would not otherwise compile correctly if the -shared option was enabled. Also, a new rule was added when target is compiled (last step of the compilation), because main.c required -lgcov. It is a relatively recent issue, and we have found no reason for why this happens when --profopt and -shared are enabled (we checked multiple times that we do not do coverage, but still we were unable to reason for this error) so we just bypassed it.

The second rule added to make, is the profopt option itself, which makes the target with profile generation enabled, then the profiles using CPython's proposed regression tests, cleans the objects, and rebuilds everything.

  • I have also changed the two profopt tests in the rpython/translator/c/test/test_standalone.py, to identify if the target was successfully created, whether -shared is enabled or not. (also the entire suite of test_standalone passed).

Performance gains:

I have run the PyPy proposed benchmark(https://bitbucket.org/pypy/benchmarks/src), and I have determined that the profile guided optimizations show an improvement of 5-6% on average. However, it is worth mentioning that the n-body simulation was particularly slowed down, by 45%. Also changing the training workload for profopt, with the benchmark itself, showed very little overall improvement, and none for n-body.

Please find the results at: https://docs.google.com/spreadsheets/d/1IYtaBCb7Lfw8vP0uiMKawxf9QfeinSzxN0YOcbkCo1o/edit#gid=1133279632

Merge request reports