Skip to content
Snippets Groups Projects
  1. Feb 21, 2025
    • Mitchell Kember's avatar
      rust: enable workspace lints · 1ef08a0381a0
      Mitchell Kember authored
      This means that lints configured in rust/Cargo.toml will apply to all crates
      within the workspace. Currently there are none but I plan to add some.
      1ef08a0381a0
  2. Feb 13, 2025
    • Raphaël Gomès's avatar
      index: remember the generaldelta config instead of getting it from the revlog · 8de68446a5bd
      Raphaël Gomès authored
      The code (especially the Rust code) was jumping in 4 dimensions to make sense
      of what was going on because it wrongly assumed that we needed to somehow
      be able to ask a generaldelta index for a non-generaldelta delta chain, which
      doesn't make any sense.
      
      Removing the cargo-culted/vestigial code, this is cleaner and less confusing.
      8de68446a5bd
  3. Jan 29, 2025
    • Georges Racinet's avatar
      rust-pyo3-dirstate: making bytes slices in core Sync · d9d6ae9b9722
      Georges Racinet authored
      For the purposes of providing PyO3 bindings, the data fields that will be
      exposed to Python  have to be `Sync`, hence that is the case of
      `OwningDirstateMap` and its `owner` field.
      
      We had to do something similar for the PyO3 bindings of `revlog`.
      
      In this case, it forces us to adapt the `Deref` wrapper of `PyBytes` used
      in `hg-cpython`, because it must itself now be `Sync` and raw pointers are
      not.
      
      This looks even uglier than it used to, but it does not matter much, because
      our ultimate goal is to remove the rust-cpython bindings altogether.
      d9d6ae9b9722
  4. Jan 07, 2025
  5. Feb 07, 2025
    • Mitchell Kember's avatar
      rust: add GraphError::ParentOutOfOrder · 2fb13c3f4496
      Mitchell Kember authored
      This will be used in a follow-up commit that creates a data structure optimized
      for inserting revisions in descending order, since it will need to fail if a
      revision number is greater than its descendant (meaning the graph is corrupted).
      2fb13c3f4496
  6. Nov 30, 2024
    • Georges Racinet's avatar
      rust-pyo3: retrieving the InnerRevlog of hg-cpython · c2480ac4c5e2
      Georges Racinet authored
      This allows PyO3-based code to use the InnerRevlog, access its shared data
      (core InnerRevlog), which will then allow, e.g., to retrieve references on
      the core Index.
      
      On the `hg-cpython` (`rusthg` crate, `rustext` Python extension module),
      we had to also build as a Rust library, and open up some accesses (see
      notably the public accessor for `inner`, the core `InnerRevlog`).
      
      Retrieving the Rust struct underlying a Python object defined by another
      extension module written in Rust is tricky because the Python type objects
      are duplicated in the extension modules, leading to failure of the normal
      type checking. See the doc-comment of `convert_cpython::extract_inner_revlog`
      for a complete explanation.
      To solve this, we import the Python type object of `rustext` (defined
      by `hg-cpython`) and perform a manual check. Checking the Python type is
      necessary, as PyO3 documentation clearly state that downcasting an object
      that has not the proper type is Undefined Behaviour.
      
      At this point, we do not have conversion facilities for exceptions (`PyErr`
      on both sides), hence the remaining unwraps).
      c2480ac4c5e2
  7. Dec 05, 2024
    • Georges Racinet's avatar
      rust: made the crate of hg-cpython importable · 15011324a80b
      Georges Racinet authored
      The crate name is actually `rusthg`. This has the side effect
      of running the doctest of the `py_shared_iterator` macro which
      was really inconsistent, and after basic fixes, exposed that the
      macro itself was poorly scoped.
      15011324a80b
  8. Nov 29, 2024
    • Georges Racinet's avatar
      rust-cpython: fix discrepancy in internal FFI lib version · 544b9d3075f4
      Georges Racinet authored
      The dependency to `sys-python3` stayed at version 0.7.1 when `cpython`
      got bumped to 0.7.2 for Python 3.12 support. In pratice, this does not
      change much because Cargo rules imply that this means ">= 0.7.1, <0.8.0".
      
      Still it did not feel right, either `cpython` enforces a stricter version
      and this specification is not needed at all, or it meant that it was still
      possible to install the older version of `sys-python3`.
      544b9d3075f4
  9. Nov 04, 2024
  10. Oct 29, 2024
  11. Nov 12, 2024
    • Raphaël Gomès's avatar
      rust-update: handle SIGINT from long-running update threads · 96b113d22b34
      Raphaël Gomès authored
      The current code does not respond to ^C until after the Rust bit is finished
      doing its work. This is expected, since Rust holds the GIL for the duration
      of the call and does not call `PyErr_CheckSignals`. Freeing the GIL to do our
      work does not really improve anything since the Rust threads are still going,
      and the only way of cancelling a thread is by making it cooperate.
      
      So we do the following:
      	- remember the SIGINT handler in hg-cpython and reset it after the call
      	  into core (see inline comment in `update.rs` about this)
      	- make all update threads watch for a global `AtomicBool` being `true`,
      	  and if so stop their work
      	- reset the global bool and exit early (i.e. before writing the dirstate)
      	- raise SIGINT from `hg-cpython` if update returns `InterruptReceived`
      96b113d22b34
  12. Nov 05, 2024
  13. Oct 10, 2024
  14. Oct 08, 2024
  15. Sep 26, 2024
  16. Jul 31, 2024
    • Raphaël Gomès's avatar
      rust-revlog: build an in-memory nodemap if a given revlog gets queried a lot · 1032bb0ef365
      Raphaël Gomès authored
      This will help with non-persistent nodemap repos that would benefit from
      one, and mirrors what the C implementation does.
      1032bb0ef365
    • Raphaël Gomès's avatar
      rust-revlog: generalize an error message · c90e0f65896e
      Raphaël Gomès authored
      This is used for more than the nodemap data.
      c90e0f65896e
    • Raphaël Gomès's avatar
      rust-revlog: don't create an in-memory nodemap for filelogs from Python · bcd4962e0df9
      Raphaël Gomès authored
      Explanations inline.
      
      Benchmarks from this change affect positively the only repo that showed this
      being a problem:
      
      ```
      ### data-env-vars.name            = mozilla-try-2024-03-26-ds2-pnm
        # benchmark.name                = hg.command.cat
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.files      = all-root
        # benchmark.variants.output     = plain
        # benchmark.variants.rev        = tip
      default:           62.848869  ~~~~~
      before-this-patch: 58.113051   (-7.54%, -4.74)
      this-patch:        57.407533   (-8.66%, -5.44)
      ### data-env-vars.name            = mozilla-try-2024-03-26-ds2-pnm
        # benchmark.name                = hg.command.log
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.limit-rev  = 10
        # benchmark.variants.patch      = yes
        # benchmark.variants.rev        = none
      default:           3.173532  ~~~~~
      before-this-patch: 3.543591  (+11.66%, +0.37)
      this-patch:        3.297235   (+3.90%, +0.12)
      ```
      bcd4962e0df9
  17. Jul 29, 2024
    • Raphaël Gomès's avatar
      revlog: add glue to use a pure-Rust VFS · 72bc29f01570
      Raphaël Gomès authored
      This will save us a lot of calling back into Python, which is always
      horribly expensive.
      
      We are now faster in all benchmarked cases except for `log --patch`
      specifically on mozilla-try. Fixing this will happen in a later patch.
      
      ```
      ### data-env-vars.name            = mercurial-devel-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.cat
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.files      = all-root
        # benchmark.variants.output     = plain
        # benchmark.variants.rev        = tip
      e679697a6ca4: 1.760765  ~~~~~
      5559d7e63ec3: 1.555513  (-11.66%, -0.21)
      ### data-env-vars.name            = mozilla-try-2024-03-26-ds2-pnm
        # benchmark.name                = hg.command.cat
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.files      = all-root
        # benchmark.variants.output     = plain
        # benchmark.variants.rev        = tip
      e679697a6ca4: 62.848869  ~~~~~
      5559d7e63ec3: 58.113051   (-7.54%, -4.74)
      ### data-env-vars.name            = mozilla-try-2024-03-26-ds2-pnm
        # benchmark.name                = hg.command.log
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.limit-rev  = 10
        # benchmark.variants.patch      = yes
        # benchmark.variants.rev        = none
      e679697a6ca4: 3.173532  ~~~~~
      5559d7e63ec3: 3.543591  (+11.66%, +0.37)
      ### data-env-vars.name            = mozilla-try-2024-03-26-ds2-pnm
        # benchmark.name                = hg.command.log
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.limit-rev  = 1000
        # benchmark.variants.patch      = no
        # benchmark.variants.rev        = none
      e679697a6ca4: 1.214698  ~~~~~
      5559d7e63ec3: 1.192478   (-1.83%, -0.02)
      ### data-env-vars.name            = mozilla-unified-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.cat
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.files      = all-root
        # benchmark.variants.output     = plain
        # benchmark.variants.rev        = tip
      e679697a6ca4: 56.205474  ~~~~~
      5559d7e63ec3: 51.520074   (-8.34%, -4.69)
      ### data-env-vars.name            = mozilla-unified-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.log
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.limit-rev  = 10
        # benchmark.variants.patch      = yes
        # benchmark.variants.rev        = none
      e679697a6ca4: 2.105419  ~~~~~
      5559d7e63ec3: 2.051849   (-2.54%, -0.05)
      ### data-env-vars.name            = mozilla-unified-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.log
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.limit-rev  = 1000
        # benchmark.variants.patch      = no
        # benchmark.variants.rev        = none
      e679697a6ca4: 0.309960  ~~~~~
      5559d7e63ec3: 0.299035   (-3.52%, -0.01)
      ### data-env-vars.name            = tryton-public-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.cat
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.files      = all-root
        # benchmark.variants.output     = plain
        # benchmark.variants.rev        = tip
      e679697a6ca4: 1.849832  ~~~~~
      5559d7e63ec3: 1.805076   (-2.42%, -0.04)
      ### data-env-vars.name            = tryton-public-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.log
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.limit-rev  = 10
        # benchmark.variants.patch      = yes
        # benchmark.variants.rev        = none
      e679697a6ca4: 0.289521  ~~~~~
      5559d7e63ec3: 0.279889   (-3.33%, -0.01)
      ### data-env-vars.name            = tryton-public-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.log
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.limit-rev  = 1000
        # benchmark.variants.patch      = no
        # benchmark.variants.rev        = none
      e679697a6ca4: 0.332270  ~~~~~
      5559d7e63ec3: 0.323324   (-2.69%, -0.01)
      ```
      72bc29f01570
    • Raphaël Gomès's avatar
      hg-core: add a complete VFS · 7be39c5110c9
      Raphaël Gomès authored
      This will be used from Python in a later change.
      
      More changes are needed in hg-core and rhg to properly clean up the APIs
      of the old VFS implementation but it can be done when the dust settles
      and we start adding more functionality to the pure Rust VFS.
      7be39c5110c9
  18. Jun 19, 2024
    • Raphaël Gomès's avatar
      revlog: add the glue to use the Rust `InnerRevlog` from Python · 7346f93be7a4
      Raphaël Gomès authored
      The performance of this has been looked at for quite some time, and some
      workflows are actually quite a bit faster than with the Python + C code.
      
      However, we are still (up to 20%) slower in some crucial places like cloning
      certain repos, log, cat, which makes this an incomplete rewrite. This is
      mostly due to the high amount of overhead in Python <-> Rust FFI, especially
      around the VFS code. A future patch series will rewrite the VFS code in
      pure Rust, which should hopefully get us up to par with current perfomance,
      if not better in all important cases.
      
      This is a "save state" of sorts, as this is a ton of code, and I don't want
      to pile up even more things in a single review.
      
      Continuing to try to match the current performance will take an extremely
      long time, if it's not impossible, without the aforementioned VFS work.
      7346f93be7a4
  19. Oct 10, 2024
  20. Oct 01, 2024
    • Raphaël Gomès's avatar
      update: add a Rust fast-path when updating from null (and clean) · 8b7123c8947b
      Raphaël Gomès authored
      This case is easy to detect and we have all we need to generate a valid
      working copy and dirstate entirely in Rust, which speeds things up
      considerably:
      
      On my machine updating a repo of ~300k files goes from 10.00s down to 4.2s,
      all while consuming 50% less system time, with all caches hot.
      Something to note is that further improvements will probably happen
      with the upcoming `InnerRevlog` series that does smarter
      mmap hanlding, especially for filelogs.
      
      Here are benchmark numbers on a machine with only 4 cores (and no SMT enabled)
      
      ```
      ### data-env-vars.name               = heptapod-public-2024-03-25-ds2-pnm
        # benchmark.name                   = hg.command.update
        # bin-env-vars.hg.py-re2-module    = default
        # bin-env-vars.hg.changeset.node   = <this change>
        # benchmark.variants.atomic-update = no
        # benchmark.variants.scenario      = null-to-tip
        # benchmark.variants.worker        = default
      default: 5.328762  ~~~~~
      rust: 1.308654  (-75.44%, -4.02)
      ### data-env-vars.name               = mercurial-devel-2024-03-22-ds2-pnm
        # benchmark.name                   = hg.command.update
        # bin-env-vars.hg.py-re2-module    = default
        # bin-env-vars.hg.changeset.node   = <this change>
        # benchmark.variants.atomic-update = no
        # benchmark.variants.scenario      = null-to-tip
        # benchmark.variants.worker        = default
      default: 1.693271  ~~~~~
      rust: 1.151053  (-32.02%, -0.54)
      ### data-env-vars.name               = mozilla-unified-2024-03-22-ds2-pnm
        # benchmark.name                   = hg.command.update
        # bin-env-vars.hg.py-re2-module    = default
        # bin-env-vars.hg.changeset.node   = <this change>
        # benchmark.variants.atomic-update = no
        # benchmark.variants.scenario      = null-to-tip
        # benchmark.variants.worker        = default
      default: 38.901613  ~~~~~
      rust: 11.637880 (-70.08%, -27.26)
      ### data-env-vars.name               = netbsd-xsrc-public-2024-09-19-ds2-pnm
        # benchmark.name                   = hg.command.update
        # bin-env-vars.hg.py-re2-module    = default
        # bin-env-vars.hg.changeset.node   = <this change>
        # benchmark.variants.atomic-update = no
        # benchmark.variants.scenario      = null-to-tip
        # benchmark.variants.worker        = default
      default: 4.793727  ~~~~~
      rust: 1.505905  (-68.59%, -3.29)
      ```
      8b7123c8947b
  21. Oct 03, 2024
  22. Oct 02, 2024
  23. Oct 01, 2024
  24. Sep 30, 2024
    • Raphaël Gomès's avatar
      rust-dirstate: use a struct as arguments for the high-level `reset_state` · 0cd16b1d613f
      Raphaël Gomès authored
      This makes the interface a lot clearer at the call site and prevents silly
      mistakes, as an API with a bunch of booleans is prone to errors.
      
      This refactor adds a `from_empty` parameter for a fast-path when resetting
      and entry we're sure does not exist. It will be used in the upcoming update
      Rust fastpath, and was not split to prevent more churn.
      0cd16b1d613f
    • Raphaël Gomès's avatar
      rust-dirstate: actually remember the identity · 88aa21d654e5
      Raphaël Gomès authored
      This was an oversight that likely had no influence on anything since we have
      only been writing the dirstate for status. We will start writing the dirstate
      for more operations now, so we'll need this fixed.
      88aa21d654e5
  25. Oct 09, 2024
  26. Sep 27, 2024
    • Pierre-Yves David's avatar
      head-revs: add a native implementation of the `stop_rev` parameter · 609700e5d8df
      Pierre-Yves David authored
      This does not add too much complexity to the native code and help with
      branchmap v3 performance.
      
      Note that the final conversion of the heads from native-code to Python is still
      too costly, especially in Rust. In addition the current caching around headrevs
      is too simple and fragile. However these are an unrelated problem.
      
      
      ### benchmark.name                                 = hg.command.unbundle
        # bin-env-vars.hg.py-re2-module                  = default
        # benchmark.variants.issue6528                   = disabled
        # benchmark.variants.resource-usage              = default
        # benchmark.variants.reuse-external-delta-parent = yes
        # benchmark.variants.revs                        = any-1-extra-rev
        # benchmark.variants.source                      = unbundle
        # benchmark.variants.validate                    = default
        # benchmark.variants.verbosity                   = quiet
      
       ## data-env-vars.name     = netbeans-2018-08-01-zstd-sparse-revlog
        # bin-env-vars.hg.flavor = default
      branch-v2:        0.233711  ~~~~~
      branch-v3 before: 0.239857   (+2.63%, +0.01)
      branch-v3 after:  0.239558   (+2.50%, +0.01)
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.235230  ~~~~~
      branch-v3 before: 0.240972   (+2.44%, +0.01)
      branch-v3 after:  0.239917   (+1.99%, +0.00)
       ## data-env-vars.name     = netbeans-2018-08-01-ds2-pnm
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.255586  ~~~~~
      branch-v3 before: 0.268560   (+5.08%, +0.01)
      branch-v3 after:  0.262261   (+2.61%, +0.01)
      
       ## data-env-vars.name     = mozilla-central-2024-03-22-zstd-sparse-revlog
        # bin-env-vars.hg.flavor = default
      branch-v2:        0.339010  ~~~~~
      branch-v3 before: 0.349389   (+3.06%, +0.01)
      branch-v3 after:  0.348247   (+2.72%, +0.01)
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.346525  ~~~~~
      branch-v3 before: 0.355661   (+2.64%, +0.01)
      branch-v3 after:  0.350906   (+1.26%, +0.00)
       ## data-env-vars.name     = mozilla-central-2024-03-22-ds2-pnm
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.380202  ~~~~~
      branch-v3 before: 0.408851   (+7.54%, +0.03)
      branch-v3 after:  0.406511   (+6.92%, +0.03)
      
      
       ## data-env-vars.name     = mozilla-unified-2024-03-22-zstd-sparse-revlog
        # bin-env-vars.hg.flavor = default
      branch-v2:        0.412165  ~~~~~
      branch-v3 before: 0.427782   (+3.79%, +0.02)
      branch-v3 after:  0.422595   (+2.53%, +0.01)
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.412397  ~~~~~
      branch-v3 before: 0.422354   (+2.41%, +0.01)
      branch-v3 after:  0.421079   (+2.11%, +0.01)
       ## data-env-vars.name     = mozilla-unified-2024-03-22-ds2-pnm
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.429501  ~~~~~
      branch-v3 before: 0.443197   (+3.19%, +0.01)
      branch-v3 after:  0.449432   (+4.64%, +0.02)
      
       ## data-env-vars.name     = mozilla-try-2024-03-26-zstd-sparse-revlog
        # bin-env-vars.hg.flavor = default
      branch-v2:        3.403171  ~~~~~
      branch-v3 before: 3.819477  (+12.23%, +0.42)
      branch-v3 after:  3.658482   (+7.50%, +0.26)
        # bin-env-vars.hg.flavor = rust
      branch-v2:        3.454876  ~~~~~
      branch-v3 before: 3.590284   (+3.92%, +0.14)
      branch-v3 after:  3.545843   (+2.63%, +0.09)
       ## data-env-vars.name     = mozilla-try-2024-03-26-ds2-pnm
        # bin-env-vars.hg.flavor = rust
      branch-v2:        3.465435  ~~~~~
      branch-v3 before: 3.633278   (+4.84%, +0.17)
      branch-v3 after:  3.556074   (+2.62%, +0.09)
      609700e5d8df
  27. Sep 26, 2024
Loading