float(x) slower on pypy3.8, pypy3.9
From the mailing list:
I have an embedded function:
@ffi.def_extern()
def toint(input,insize,result):
for i in range(insize):
x = ffi.string(input[i])
result[i] = float(x) if x else 0
return 1
It seems that this runs around 30-40% percent slower in PyPy3 than in 2.
Input is a char** c array and result is a float c array. The difference is not in ffi.string If having
result[i] = 1 if x else 0
it runs same in both versions. Any ideas?
I think there is something slower with float(str)
. If I try this script
import time
def f(invals, conv):
res = 0
start = time.time()
for i in range(4000):
for val in invals:
res += conv(val)
stop = time.time()
print(res, stop - start)
f([str(x) for x in range(10000)], float)
f(range(10000), float)
f([str(x) for x in range(10000)], int)
f(range(10000), int)
I get these timings
version | float(str) | float(int) | int(str) | int(int) |
---|---|---|---|---|
pypy2.7 | 2290ms | 150ms | 406ms | 170ms |
pypy3.7 | 3450ms | 130ms | 430ms | 140ms |
pypy3.8 | 3890ms | 130ms | 510ms | 140ms |
pypy3.9 | 4010ms | 135ms | 510ms | 140ms |
cpython2.7 | 3820ms | 2980ms | 8300ms | 2670ms |
cpython3.8 | 3410ms | 2020ms | 3740ms | 1970ms |
From a quick glance at W_FloatObject.descr__new__
and the helper function unicode_to_decimal_w
I see:
- Python3 checks for both
__float__
and__index__
before doing a conversion fromstr
, Python2 checks only for__float__
- Python3 has a fast path in
unicode_to_decimal_w
whenw_unistr.is_ascii()
, which should make it faster than Python2 not slower - Python3 adds a
_remove_underscores
pass on the utf8 string.
I tried disabling the _remove_underscore
call and got these timings for py3.7
version | float(str) | float(int) | int(str) | int(int) |
---|---|---|---|---|
pypy3.7 no _remove_underscore
|
2420ms | 130ms | 430ms | 140ms |
so it seems the _remove_undersore
call is relatively expensive in this microbenchmark, probably because everything else is so "inexpensive". We could do some work to minimize the number of passes we make across the string:
- in floatobject
- check for ascii (boolean result)
- check for underscores and remove them
- in rfloat
- check for spaces and take only the first chunk
- lower case the string