Created originally on Bitbucket by kostialopuhin (Konstantin Lopukhin)
Was already merged in Bitbucket before import, marked as merged by the import user
As noted in https://bugs.pypy.org/issue979 pickle.dumps is much slower than pickle.dump, because of slow StringIO. It would be better to fix StringIO instead, and I will try, but for now this patch fixes pickle.dumps slowness by using StringBuilder instead. After this change pickle.dumps has the same speed as pickle.dump, I also run pickle app-level tests
This is PR #131 (closed) redone after advice from Brian Kearns. Now it is faster both on small and large objects:
#!python
class Point(object):
def __init__(self, x, y):
self.x = x
self.y = y
def test():
n_loops = 1
N = 500
data = [Point(x, y) for x in xrange(N) for y in xrange(N)]
for _ in xrange(n_loops):
len(pickle.dumps(data))
N = 500, n_loops = 1
#!
3.17 s vs 22.9 s on default
2.13 s vs 8.07 s on default - with HIGHEST_PROTOCOL
N = 2, n_loops = 200000
#!
4.63 s vs 5.02 on default
N = 2, n_loops = 2000000
#!
3.53 s vs 3.64 on default - with HIGHEST_PROTOCOL
And for interegs instead of instances:
N = 1000, n_loops = 10
#!
1.86 vs 24.0 on default
1.45 vs 5.15 on default - with HIGHEST_PROTOCOL
N = 2, n_loops = 200000
#!
2.34 s vs 3.80 on default
0.34 s vs 0.54 on default - with HIGHEST_PROTOCOL