PyPy should set `errors="surrogatepass"` when decoding numpy arrays to unicode scalars
Created originally on Bitbucket by ericwieser (Eric Wieser)
See https://github.com/numpy/numpy/issues/15363, where the bug was reported in numpy.
The fix in numpy was to replace calls to PyUnicode_DecodeUTF32
with calls to PyUnicode_FromKindAndData(PyUnicode_4BYTE_KIND, ...)
.
However, it seems that PyPI does not use our code there, and has their own implementation. Assuming PyUnicode_FromKindAndData
is not available, the equivalent python code would be some_bytes.decode('utf-32-le', errors='surrogatepass')
.