iter(open(filename)) very slow in PyPy3
Created originally on Bitbucket by antocuni (Antonio Cuni)
Consider the following benchmark:
import sys
import time
def main():
fname = sys.argv[1]
N = 100
a = time.time()
for i in range(N):
with open(fname, 'rb') as f:
for line in f:
pass
b = time.time()
t = (b-a) * 1000.0 / N
print('%6.2f ms per iteration [%2d iterations]' % (t, N))
main()
I ran it on this (unzipped) file: https://data.gharchive.org/2015-01-01-15.json.gz on bencher4. These are the results:
branch | rev | ms | ms (--jit off) |
---|---|---|---|
py3.6 | df95b7b0b5c6 | 41.76 | 44.98 |
release-pypy3.6-v7.2.0 | 5da45ced70e5 | 64.54 | 67.84 |
pypy3 7.2.0 portable | 5da45ced70e5 | 39.81 | 43.29 |
hpy | 95b626a1c6e0 | 64.43 | 68.04 |
pypy2 7.2.0 portable | 4a68d8d3d2fc | 17.12 | 18.02 |
CPython 2.7.11 | 7.00 | ||
CPython 3.7.5 | 13.39 |
Repeating the benchmark multiple times didn’t change much the numbers. It seems there is a huge variation between different builds (including portable vs non-portable). And it seems to be much slower than PyPy2 and the two CPythons.