str.encode("utf-8") does not use O(1) fast path
I’m using PyPy3 v7.3.9 (installed from the Arch Linux repos) and noticed that using str.encode
with the encoding utf-8
is slow on large strings.
$ pypy3 --version
Python 3.8.13 (4b1398fe9d76ad762155d03684c2a153d230b2ef, Apr 02 2022, 15:38:25)
[PyPy 7.3.9 with GCC 11.2.0]
utf8
, UTF8
and UTF-8
(and even utf-⛄
) all use the O(1) fast path:
In [1]: s = "1234" * 12345678
In [5]: %timeit s.encode("utf8")
1.66 ns ± 0.0108 ns per loop (mean ± std. dev. of 7 runs, 1,000,000,000 loops each)
In [6]: %timeit s.encode("utf-8")
47.4 ms ± 590 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [7]: import sys
In [8]: sys.getdefaultencoding()
Out[8]: 'utf-8'
In [9]: %timeit s.encode()
48 ms ± 816 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [10]: %timeit s.encode("UTF8")
1.66 ns ± 0.00558 ns per loop (mean ± std. dev. of 7 runs, 1,000,000,000 loops each)
In [11]: %timeit s.encode("UTF-8")
1.63 ns ± 0.0998 ns per loop (mean ± std. dev. of 7 runs, 1,000,000,000 loops each)
In [13]: %timeit s.encode("utf-⛄")
1.6 ns ± 0.0132 ns per loop (mean ± std. dev. of 7 runs, 1,000,000,000 loops each)
This also reproduces on the latest nightly:
$ ./bin/pypy3 --version
Python 3.9.12 (8bb6bdff248b99ec4bc65da1fbad0d774b5d3316, May 30 2022, 01:38:45)
[PyPy 7.3.10-alpha0 with GCC 10.2.1 20210130 (Red Hat 10.2.1-11)]
Am I missing something or should utf-8
be handled the same as all the other spellings?