pypy3.6: PyUnicode Api in PyPy has unexpected behavior
In a Python C extension I use the macros:
PyUnicode_KIND,
PyUnicode_DATA,
PyUnicode_GET_LENGTH
to get the internal buffer of a string. E.g. when the kind is PyUnicode_1BYTE_KIND I treat the pointer returned by PyUnicode_DATA as a uint8_t* pointer and then read length elements from there. However in PyPy this does not work.
An example for this is the string '000\x80' in Cpython this has the following data:
kind: 1
length: 4
data: 48 48 48 128
In PyPy on the other hand the API returns the following data:
kind: 1
length: 4
data: 48 48 48 194 128
(In my application I would not read the 128, since it is placed after the length of the string)
My first thought was that I might have to use PyUnicode_READ in PyPy and are not allowed to use the buffer directly. However the same behavior occurs when reading with PyUnicode_READ or PyUnicode_READ_CHAR
To clarify the PyPy version I am using: Python 3.6.9 (4d5bb11d5832, Aug 02 2020, 15:14:33) [PyPy 7.3.1 with GCC 10.2.1 20200723 (Red Hat 10.2.1-1)] on linux
Could someone clarify, whether this is a bug, or how this API should be used correctly in PyPy? Thanks in advance