Skip to content
Snippets Groups Projects
  1. Feb 21, 2025
    • Mitchell Kember's avatar
      rust: enable workspace lints · 1ef08a0381a0
      Mitchell Kember authored
      This means that lints configured in rust/Cargo.toml will apply to all crates
      within the workspace. Currently there are none but I plan to add some.
      1ef08a0381a0
  2. Feb 07, 2025
    • Mitchell Kember's avatar
      rust-ancestors: use BitSet for seen revisions · e9ced84aeef4
      Mitchell Kember authored
      This makes AncestorsIterator store its set of seen revisions in a BitSet-based
      DescendingRevisionSet instead of a HashSet. This provides faster lookups and
      insertions, and uses only A - B bits of memory when iteration goes from revision
      A down to revision B. In the worst case iterating from tip to -1 in the
      mercurial-devel changelog, for example, it would use about 7 KiB.
      
      Running rhg annotate on 200 random files in mercurial-devel gave on average a
      10% improvement. Here is the distribution:
      
       new/old  freq    histogram
      -------- |----- | ---------
          0.81 |   17 | **
          0.84 |   32 | ****
          0.87 |   23 | ***
          0.90 |   24 | ***
          0.92 |   20 | ***
          0.95 |   34 | *****
          0.98 |   18 | **
          1.01 |   24 | ***
          1.04 |    3 |
          1.07 |    5 |
      -------- |----- | ---------
      Avg=0.90 |N=200 |
      e9ced84aeef4
  3. Feb 05, 2025
    • Mitchell Kember's avatar
      rust: add encoding.rs · bbf1c52252ae
      Mitchell Kember authored
      This is based on encoding.py. It reads the environment variables HGENCODING,
      HGENCODINGMODE, and HGENCODINGAMBIGUOUS. Currently it only supports UTF-8 and
      ascii, but it could be extended to support other local encodings.
      
      Unlike Python, it assumes all internal strings are UTF-8 and does not attempt to
      fallback to latin-1 (or ui.fallbackencoding).
      
      Nothing is using this now, but in the future command output and error messages
      should transition to using it.
      
      I replaced existing calls to `utf8_to_local` and `local_to_uf8` with direct
      String/bytes methods since they were not logically converting between internal
      and local encodings. Instead, they were used (for example) when an error message
      happened to be stored as String but needed to be passed somewhere as bytes. The
      proper fix for this will be to avoid String in the first place.
      bbf1c52252ae
  4. Dec 18, 2024
    • Mitchell Kember's avatar
      rust: add safe bindings to bdiff.c · 1b7a57a5b47a
      Mitchell Kember authored
      I wrote C FFI bindings manually rather than using a bindgen build step because
      there are only 2 structs and 3 functions and they're not going to change.
      
      Note that the relative path in build.rs means that cargo publish will no longer
      work. If in the future we want to publish to crates.io, we would probably need
      to add a Makefile step that copies bdiff sources into the hg-core crate.
      1b7a57a5b47a
  5. Dec 06, 2024
    • Arseniy Alekseyev's avatar
      rust-ignore: construct regex Hir object directly, avoiding large regex string · 1866119cbad7
      Arseniy Alekseyev authored
      Rework how we convert patterns to regexes in rust.
      Instead of going patterns -> string -> Regex, which is slow and causes
      some correctness issues, build a structured regex_syntax::hir::Hir value,
      which is faster and it also prevents surprising regex escape.
      
      This change makes the time of `build_regex_match` go from ~70-80ms
      to ~40ms in my testing (for a large hgignore).
      
      The bug I mentioned involves regex patterns that "escape" their
      intended scope. For example, a sequence of hgignore regexp patterns like
      this would previously lead to surprising behavior:
      
          foo(?:
          bar
          baz
          )
      
      this matches foobar and foobaz, and doesn't match bar and baz.
      
      The new behavior is to report a pattern parse error
      The Python hg also has this bug, so this bugfix
      not really helping much, but it's probably better to
      fall back to real Python bugs than to simulate them.
      1866119cbad7
  6. Nov 04, 2024
  7. Nov 12, 2024
    • Raphaël Gomès's avatar
      rust-manifest: use `memchr` crate for all byte-finding needs · f4aede0f01af
      Raphaël Gomès authored
      While writing a very dumb manifest diffing algorithm for a proof-of-concept
      I saw that `Manifest::find_by_path` was much slower than I was expecting.
      
      It turns out that the Rust stdlib uses slow (all is relative) code when
      searching for byte positions for reasons ranging from portability, SIMD
      API stability, nobody doing the work, etc. `memch` is much faster for these
      purposes, so let's use it.
      
      I was measuring ~670ms of profile time in `find_by_path`, after this patch
      it went down to ~230ms.
      f4aede0f01af
  8. Nov 14, 2024
  9. Nov 12, 2024
    • Raphaël Gomès's avatar
      rust-update: handle SIGINT from long-running update threads · 96b113d22b34
      Raphaël Gomès authored
      The current code does not respond to ^C until after the Rust bit is finished
      doing its work. This is expected, since Rust holds the GIL for the duration
      of the call and does not call `PyErr_CheckSignals`. Freeing the GIL to do our
      work does not really improve anything since the Rust threads are still going,
      and the only way of cancelling a thread is by making it cooperate.
      
      So we do the following:
      	- remember the SIGINT handler in hg-cpython and reset it after the call
      	  into core (see inline comment in `update.rs` about this)
      	- make all update threads watch for a global `AtomicBool` being `true`,
      	  and if so stop their work
      	- reset the global bool and exit early (i.e. before writing the dirstate)
      	- raise SIGINT from `hg-cpython` if update returns `InterruptReceived`
      96b113d22b34
  10. Oct 10, 2024
    • Raphaël Gomès's avatar
      rust-revlog: add a Rust-only `InnerRevlog` · e01e84e5e426
      Raphaël Gomès authored
      This mirrors the Python `InnerRevlog` and will be used in a future patch
      to replace said Python implementation. This allows us to start doing more
      things in pure Rust, in particular reading and writing operations.
      
      A lot of changes have to be introduced all at once, it wouldn't be very
      useful to separate this patch IMO since all of them are either interlocked
      or only useful with the rest.
      e01e84e5e426
  11. Oct 01, 2024
    • Raphaël Gomès's avatar
      update: add a Rust fast-path when updating from null (and clean) · 8b7123c8947b
      Raphaël Gomès authored
      This case is easy to detect and we have all we need to generate a valid
      working copy and dirstate entirely in Rust, which speeds things up
      considerably:
      
      On my machine updating a repo of ~300k files goes from 10.00s down to 4.2s,
      all while consuming 50% less system time, with all caches hot.
      Something to note is that further improvements will probably happen
      with the upcoming `InnerRevlog` series that does smarter
      mmap hanlding, especially for filelogs.
      
      Here are benchmark numbers on a machine with only 4 cores (and no SMT enabled)
      
      ```
      ### data-env-vars.name               = heptapod-public-2024-03-25-ds2-pnm
        # benchmark.name                   = hg.command.update
        # bin-env-vars.hg.py-re2-module    = default
        # bin-env-vars.hg.changeset.node   = <this change>
        # benchmark.variants.atomic-update = no
        # benchmark.variants.scenario      = null-to-tip
        # benchmark.variants.worker        = default
      default: 5.328762  ~~~~~
      rust: 1.308654  (-75.44%, -4.02)
      ### data-env-vars.name               = mercurial-devel-2024-03-22-ds2-pnm
        # benchmark.name                   = hg.command.update
        # bin-env-vars.hg.py-re2-module    = default
        # bin-env-vars.hg.changeset.node   = <this change>
        # benchmark.variants.atomic-update = no
        # benchmark.variants.scenario      = null-to-tip
        # benchmark.variants.worker        = default
      default: 1.693271  ~~~~~
      rust: 1.151053  (-32.02%, -0.54)
      ### data-env-vars.name               = mozilla-unified-2024-03-22-ds2-pnm
        # benchmark.name                   = hg.command.update
        # bin-env-vars.hg.py-re2-module    = default
        # bin-env-vars.hg.changeset.node   = <this change>
        # benchmark.variants.atomic-update = no
        # benchmark.variants.scenario      = null-to-tip
        # benchmark.variants.worker        = default
      default: 38.901613  ~~~~~
      rust: 11.637880 (-70.08%, -27.26)
      ### data-env-vars.name               = netbsd-xsrc-public-2024-09-19-ds2-pnm
        # benchmark.name                   = hg.command.update
        # bin-env-vars.hg.py-re2-module    = default
        # bin-env-vars.hg.changeset.node   = <this change>
        # benchmark.variants.atomic-update = no
        # benchmark.variants.scenario      = null-to-tip
        # benchmark.variants.worker        = default
      default: 4.793727  ~~~~~
      rust: 1.505905  (-68.59%, -3.29)
      ```
      8b7123c8947b
  12. Sep 30, 2024
    • Raphaël Gomès's avatar
      rust-hg-cpython: add an `HgProgressBar` util · 92e23ba257d1
      Raphaël Gomès authored
      This will be the entry point for all progress bars from a Python context
      in upcoming patches. Like the `Progress` trait, this is subject to change
      once we have more use cases, but this is good enough for now.
      92e23ba257d1
  13. Jun 19, 2024
    • Raphaël Gomès's avatar
      rust: add Vfs trait · db7dbe6f7bb2
      Raphaël Gomès authored
      This will allow for the use of multiple vfs like in the Python implementation,
      as well as hiding the details of the upcoming Python vfs wrapper to hg-core.
      db7dbe6f7bb2
  14. Feb 15, 2024
  15. Nov 29, 2023
  16. Jul 06, 2023
    • Raphaël Gomès's avatar
      rust-config: add support for default config items · f8412da86d05
      Raphaël Gomès authored
      Now that configitems.toml exists, we can read from it the default values for
      all core config items.
      
      We will add the devel-warning for use of undeclared config items in a later
      patch when we're done adding the missing entries for `rhg`.
      f8412da86d05
  17. Jun 12, 2023
    • Raphaël Gomès's avatar
      rust-hg-core: move from `ouroboros` to `self_cell` · 2cc5de261d76
      Raphaël Gomès authored
      `ouroboros` has a fundamental soundness problem that, while not applicable
      today, could become applicable given new compiler optimizations.¹
      
      `self_cell` is a crate that accomplishes a lot of the same things that
      `ouroboros` did while remaining sound (that is, unless a new soundness issue
      is discovered) by not assuming as much about the memory layout of the program.
      `self_cell` has been scrutinized heavily in the past few months by very
      competent people, some from the compiler team and has shown no weaknesses
      for a while, with a 1.0 stable release coming out a couple months ago.
      
      Our internal API is exactly the same, this is just an implementation detail.
      To reiterate, no actual soundness issue was found with our use of `ouroboros`,
      but there might be evolutions of `rustc` (or even a future separate compiler)
      that could generate unsound code.
      
      [1] https://github.com/joshua-maros/ouroboros/issues/88
      2cc5de261d76
  18. Mar 06, 2023
  19. Feb 22, 2023
  20. Jan 10, 2023
  21. Jan 06, 2023
    • Raphaël Gomès's avatar
      rust: use `logging_timer` instead of `micro_timer` · c15b415d1bff
      Raphaël Gomès authored
      I am the author of `micro_timer`.
      I built it at the time because I couldn't find a crate that was simple to use
      and flexible to do function timing with. Turns out I just couldn't find it
      because crates.io's search isn't all that great, or maybe I didn't look hard
      enough.
      
      `logging_timer` is better in every way:
          - supports changing the logging level
          - supports start and end logging
          - supports intermediary messages
          - supports inline macros
          - supports formatting the output
          - better IDE/tree-sitter integration thanks to a more robust proc macro
      
      I also changed all uses to one-liners, so it's easier to copy-paste.
      c15b415d1bff
  22. Nov 14, 2022
  23. Nov 10, 2022
  24. Oct 10, 2022
    • Arseniy Alekseyev's avatar
      dirstate-v2: skip evaluation of hgignore regex on cached directories · eb02decdf0ab
      Arseniy Alekseyev authored
      By making the computation of [has_ignored_ancestor] lazy we're eliding
      its computation in the common case when none of its descendants have
      changed on disk.
      
      On a ~400k files repo, with a cached status, we saw a ~64% reduction
      in CPU time, resulting in a speedup of ~10-15% (on ZFS), and a speedup
      of ~38% of XFS (XFS has faster stat operations for some reason).
      eb02decdf0ab
  25. Jul 11, 2022
    • kiilerix's avatar
      rust: bump to memmap2 0.5.3, micro-timer 0.4.0, and crossbeam-channel 0.5.0 · 1bad05cfc818
      kiilerix authored
      The merge in 12adf8c695ed had conflicts in rust/Cargo.lock and
      rust/hg-core/Cargo.toml . Let's ignore rust/Cargo.lock - it is regenerated.
      
      For rust/hg-core/Cargo.toml, stable had dd6b67d5c256 "rust: fix unsound
      `OwningDirstateMap`" which introduced ouroboros (and dropped
      stable_deref_trait).
      
      Default had ec8d9b5a5e7c "rust-hg-core: upgrade dependencies" which had a lot
      of churn bumping minimum versions - also patch versions. It is indeed a good
      idea to bump to *allow* use of latest package. That means that major versions
      should be bumped for packages after 1.0, and for packages below 1.0 minor
      versions should be bumped too. But it doesn't work to try enforce a policy of
      using latest patch by bumping versions at arbitrary times.
      
      For good or bad, the merge doesn't seem to have resolved the conflicts
      correctly, and many of the minor "upgrade dependencies" were lost again.
      
      Unfortunately, it also lost the bump of memmap2 to 0.5.3, which is needed for
      Fedora packaging where 0.4 isn't available. Same with micro-timer bump to 0.4
      (which already is used in rhg). crossbeam-channel bump was also lost.
      
      This change fixes that regression by redoing these "important" lines of the
      merge "correctly".
      
      I propose this for stable, even though dependency changes on stable branches
      are annoying.
      1bad05cfc818
  26. Jun 08, 2022
  27. May 24, 2022
    • kiilerix's avatar
      rust: relax im-rc dependency to allow minor updates · 13c37f1c7c4b
      kiilerix authored
      This "15.0.*" requirement came from 0d99778af68a and is now replaced with plain
      "15.0".
      
      AFAICS, it really should allow (but not necessarily require) im-rc 15.1 .
      
      Narrow requirement requirements with wildcard in the version is not used in
      other places.
      13c37f1c7c4b
  28. Apr 05, 2022
    • Raphaël Gomès's avatar
      merge: stable into default · 12adf8c695ed
      Raphaël Gomès authored
      12adf8c695ed
    • Raphaël Gomès's avatar
      rust: fix unsound `OwningDirstateMap` · dd6b67d5c256
      Raphaël Gomès authored
      As per the previous patch, `OwningDirstateMap` is unsound. Self-referential
      structs are difficult to implement correctly in Rust since the compiler is
      free to move structs around as much as it wants to. They are also very rarely
      needed in practice, so the state-of-the-art on how they should be done within
      the Rust rules is still a bit new.
      
      The crate `ouroboros` is an attempt at providing a safe way (in the Rust sense)
      of declaring self-referential structs. It is getting a lot attention and was
      improved very quickly when soundness issues were found in the past: rather than
      relying on our own (limited) review circle, we might as well use the de-facto
      common crate to fix this problem. This will give us a much better chance of
      finding issues should any new ones be discovered as well as the benefit of
      fewer `unsafe` APIs of our own.
      
      I was starting to think about how I would present a safe API to the old struct
      but soon realized that the callback-based approach was already done in
      `ouroboros`, along with a lot more care towards refusing incorrect structs.
      
      In short: we don't return a mutable reference to the `DirstateMap` anymore, we
      expect users of its API to pass a `FnOnce` that takes the map as an argument.
      This allows our `OwningDirstateMap` to control the input and output lifetimes
      of the code that modifies it to prevent such issues.
      
      Changing to `ouroboros` meant changing every API with it, but it is relatively
      low churn in the end. It correctly identified the example buggy modification of
      `copy_map_insert` outlined in the previous patch as violating the borrow rules.
      
      Differential Revision: https://phab.mercurial-scm.org/D12429
      dd6b67d5c256
  29. Mar 09, 2022
    • Raphaël Gomès's avatar
      rust-hg-core: upgrade dependencies · ec8d9b5a5e7c
      Raphaël Gomès authored
      This upgrades all dependencies to their latest version, except `clap` and `zstd`
      whose latest versions do not support our minimum supported Rust version 1.48.0.
      
      Same as for `rhg`, it contains security fix for `regex` which does not affect
      us too much, but doesn't hurt, and the rest of the upgrades are there simply
      to keep up.
      
      Differential Revision: https://phab.mercurial-scm.org/D12358
      ec8d9b5a5e7c
  30. Feb 08, 2022
    • Simon Sapin's avatar
      dirstate-tree: optimize HashMap lookups with raw_entry_mut · 11c0411bf4e2
      Simon Sapin authored
      This switches to using `HashMap` from the hashbrown crate,
      in order to use its `raw_entry_mut` method.
      The standard library’s `HashMap` is also based on this same crate,
      but `raw_entry_mut` is not yet stable there:
      https://github.com/rust-lang/rust/issues/56167
      
      Using version 0.9 because 0.10 is yanked and 0.11 requires Rust 1.49
      
      This replaces in `DirstateMap::get_or_insert_node` a call to
      `HashMap<K, V>::entry` with `K = WithBasename<Cow<'on_disk, HgPath>>`.
      
      `entry` takes and consumes an "owned" `key: K` parameter, in case a new entry
      ends up inserted. This key is converted by `to_cow` from a value that borrows
      the `'path` lifetime.
      
      When this function is called by `Dirstate::new_v1`, `'path` is in fact
      the same as `'on_disk` so `to_cow` can return an owned key that contains
      `Cow::Borrowed`.
      
      For other callers, `to_cow` needs to create a `Cow::Owned` and thus make
      a costly heap memory allocation. This is wasteful if this key was already
      present in the map. Even when inserting a new node this is typically the case
      for its ancestor nodes (assuming most directories have numerous descendants).
      
      Differential Revision: https://phab.mercurial-scm.org/D12317
      11c0411bf4e2
  31. Jan 14, 2022
  32. Dec 17, 2021
  33. Oct 12, 2021
    • Simon Sapin's avatar
      dirstate-v2: Replace the 32-bit `mode` field with two bits · 4d5a13253d34
      Simon Sapin authored
      Previously we stored the entire value from `stat_result.st_mode`,
      like dirstate-v1 does. However only the executable permission
      and type of file (only symbolic links and normal files are supported)
      are relevant to Mecurial.
      
      So replace this field with two bits in the existing bitfield byte.
      For now the unused space is left as padding, as it will be used
      for something else soon.
      
      Differential Revision: https://phab.mercurial-scm.org/D11635
      4d5a13253d34
Loading