tdigest with PyPy is 8x slower than with Python3 or Python2
Created originally on Bitbucket by bytenerd (OvanesM)
I ran the tests below and found out that on PyPy tdigest is horribly slow. Besides update I think all operations like merging of tdigests are slow.
I opened a similar issue in tdigest GitHub: https://github.com/CamDavidsonPilon/tdigest/issues/44.
Timings are:
==========
PyPy
running test in 2.7.13 (0e7ea4fe15e82d5124e805e2e4a37cae1a402d4b, Jan 06 2018, 12:46:49)
[PyPy 5.10.0 with GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)]
generating tdigests in batch
generated items: 100
generating tdigests took: 32.5672068596
----------
generating tdigests one by one
generated items: 100
generating tdigests took: 17.4209430218
----------
==================
Python
running test in 2.7.14 (default, Mar 9 2018, 23:57:12)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)]
generating tdigests in batch
generated items: 100
generating tdigests took: 4.16117596626
----------
generating tdigests one by one
generated items: 100
generating tdigests took: 2.38711595535
----------
I've repeated the test in the official PyPy docker container with PyPy 6.0.0 (compatible with python 3) with the same outcome: https://hub.docker.com/_/pypy/
Any ideas?
Attachments: tdigest_perf_test.py
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information