Skip to content

WIP: Rust nodemap

Raphaël Gomès requested to merge topic/default/rust-nodemap into branch/default

A first actually working version of a nodemap implemented in Rust with an append-only variant of the same 16-radix trie as in revlog.c, packed in the rustext.revlog.MixedIndex class, to instantiate with the C index.

Currently activated by the presence of the HGRUSTINDEX environment variable. It passes all the tests for me. It has the methods required for serialization by the Python layer:

  • can be instantiated on anything implementing the buffer protocal for bytes (including mmap): MixedIndex(c_idx, data, datalen)
  • can return a ready to write dump all its mutable data as a bunch of bytes ready to be appended to the existing cache
  • pack / vacuum logic can for the time being be done this way:
c_index = MixedIndex.get_cindex()
packer = MixedIndex(c_index, None, 0)
packer._fill_nodemap()
with open(nodemap_fpath, 'wb') as fobj:  
    fobj.write(packer.nodemap_writeable())

(of course that example is oversimplified, one would need vfs, locks, perform an atomic rewrite etc, but that's not the point).

Obviously not ready for submission. Lots of squashing / resplitting and clarification of implementation, API etc, but it passes the whole test suite for me with Python 2.7

Merge request reports