Created originally on Bitbucket by antocuni (Antonio Cuni)
The main goal of the branch is to make struct
faster; in particular:
-
struct.unpack
is now optimized for all kind of buffers, including bytearray and raw buffers; previously, the fast path was taken only for strings -
struct.pack_into
is now optimized: it uses native store ops whenever possible (gc_store_indexed or raw_store) and writes directly to the buffer (previously it was implemented in terms ofpack
+ setslice): I measured speedups up to 7x. -
struct.pack
is faster as well: previously it was based on StringBuilder and produced the resulting string piece-by-piece: now it preallocates a buffer of the right size and fills it using the same logic aspack_into
.
The branch contains a lot of refactorings and new features; here are what I think are the most important and/or delicate ones:
-
introduced
rlib.buffer.Buffer.typed_{read,write}
, so that each buffer type can decide if and how to implement them (e.g. by usingraw_{load,store}
orgc_{load,store}_indexed
depending on the type of underlying buffer) -
refactored
rlib/rstruct
andpypy/module/struct
to make advantage of the new buffer interface -
added
rlib/mutbuffer.py
: this preallocates and RPython string but acts like a buffer so that you can write things to it; when you are done, you callfinish()
, you the the high-level rpython str and you are forbidden to make more writes -
added
llop.gc_store_indexed
and fixed the zillion of places which needed it -
taught
writeanalyze.py
aboutllop.gc_{load,store}_indexed
: the idea is that they generate the same effect as a "normal" write to the same data (so for example,'interiorfield'
to forSTR.chars
andarray
forLIST.items
)