Skip to content
Snippets Groups Projects
  1. Jan 06, 2025
    • Matt Harbison's avatar
      pyupgrade: drop usage of py3 aliases for `OSError` · 24ee91ba9aa8
      Matt Harbison authored
      These were different classes in py2, but now a handful of error classes are just
      an alias of `OSError`, like `IOError`, `EnvironmentError`, `WindowsError`, etc.
      
      This is the result of running a hacked version of `pyupgrade` 3.19.1[1]
      
          $ hg files -0 'relglob:**.py' | xargs -0 \
              pyupgrade --py38-plus --keep-percent-format --keep-mock --keep-runtime-typing
      
      The hack is because it doesn't have command line switches to disable most
      changes, so it makes tons of unrelated changes all at once.  The hack is to
      
        1) patch `pyupgrade._main._fix_tokens()` to immediately return its content arg
        2) change `pyupgrade._data.register_decorator()` to only register the function
           if it's from the fixer we're interested in:
      
            if func.__module__ in (
                "pyupgrade._plugins.exceptions",
            ):
                FUNCS[tp].append(func)
      
            return func
      
      [1] https://github.com/asottile/pyupgrade
      24ee91ba9aa8
    • Matt Harbison's avatar
      py3: drop redundant `u''` prefixes on string literals · 9db77d46de79
      Matt Harbison authored
      Strings are unicode on Python 3.  These were rewritten by `pyupgrade`.
      
      It's arguably better to fix the `contrib` stuff upstream and then re-vendor it,
      but I don't feel like waiting for that, and then all of the regression testing
      involved to get a minor improvement in the codebase.  It was last vendored 5
      years ago, and will likely be a large change anyway to drop py2 support.  Also,
      we've already made minor formatting changes to it locally.
      9db77d46de79
    • Matt Harbison's avatar
      style: drop extraneous parentheses · dc36535a5edc
      Matt Harbison authored
      These were rewritten by `pyupgrade`.
      dc36535a5edc
    • Matt Harbison's avatar
      check-code: reference the `noi18n` methods in the missing `_()` message · 3b6f25190157
      Matt Harbison authored
      The regex is complex enough, so I'm not bothering with figuring out how to
      detect and fail the older style of wrapping the constant string in `()`.
      3b6f25190157
    • Matt Harbison's avatar
      style: use `ui.xxxnoi18n()` methods instead of wrapping msg in `()` · 0e11e532c958
      Matt Harbison authored
      These aliases were introduced back in 5209fc94b982, because `black` was going to
      strip away the extra parentheses, but they're needed to subvert `test-check-code.t`.
      That obviously changed at some point, but `pyupgrade`[1] also strips these out.
      While that tool is very useful in adapting code to modern standards, it lacks
      the ability to turn off most conversions, so constantly reverting these is a
      pain.
      
      Even without that, the code is more understandable with an explicit declaration.
      It also would have been an easy typo to miss the leading `_` in the i18n method
      `_()` that the checker is looking for, and fail to detect the problem.
      
      The `contrib/perf.py` code just uses a local alias to the original methods
      because (IIUC), this tries to be compatible with old versions of hg.  But
      practically, these noi18n aliases were added before useful py3 support, and at
      some point, it won't be feasible to do py2 benchmarking anymore, and maybe this
      module can be cleaned up some.
      
      [1] https://github.com/asottile/pyupgrade
      0e11e532c958
  2. Jan 07, 2025
    • Matt Harbison's avatar
      ci: include `test-contrib-check-*.t` in the `checks` group · cdfaba964f21
      Matt Harbison authored
      This covers `test-contrib-check-code.t` and `test-contrib-check-commit.t`.  I
      overlooked these recently when hacking on `contrib/check-code.py`, and ran
      `test-check-*` locally.  I was initially inclined to rename these
      `test-check-contrib-*.t`, but we already have `test-check-code.t` and
      `test-check-commit.t`, so this would break tab completion.  Since these test
      rarely modified tools, let's leave them as-is.  This change simply tests them
      once in CI, instead of running them for each flavor of py + modulepolicy that we
      test.
      cdfaba964f21
    • Matt Harbison's avatar
      cborutil: fix an error interpolating str into bytes · fb8932685031
      Matt Harbison authored
      Also, `ValueError` wants str anyway.
      fb8932685031
  3. Jan 04, 2025
    • Jeff Sipek's avatar
      git: hexdump node when passing it to sqlite in baselog.hasnode · 86920d1f7632
      Jeff Sipek authored
      This change makes `hg heads` work.  Prior to this change, the error was:
      
      $ hg heads
      ** unknown exception encountered, please report by visiting
      ** https://mercurial-scm.org/wiki/BugTracker
      ** Python 3.11.10 (main, Oct 31 2024, 01:10:40) [Clang 18.1.5 (https://github.com/llvm/llvm-project.git llvmorg-18.1.5-0-g617a15
      ** Mercurial Distributed SCM (version 6.9.post1.dev337+hg.72af9fa34832)
      ** Extensions loaded: absorb, churn, git (pygit2 1.14.1), githelp, gpg, hgk, histedit, patchbomb, rebase
      Traceback (most recent call last):
        File "/home/jeffpc/src/oss/hg-gitext-test/../hg-gitext/hg", line 61, in <module>
          dispatch.run()
      ...
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/commands.py", line 3462, in heads
          heads += repo.branchheads(branch, start, opts.get('closed'))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/localrepo.py", line 3469, in branchheads
          if not branches.hasbranch(branch):
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/branchmap.py", line 701, in hasbranch
          self._verifybranch(label)
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/branchmap.py", line 670, in _verifybranch
          _unknownnode(n)
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/branchmap.py", line 186, in _unknownnode
          raise ValueError('node %s does not exist' % node.hex())
      ValueError: node f2f80ac809875855ac843f9e5e7480604b5cbff5 does not exist
      86920d1f7632
    • Jeff Sipek's avatar
      git: switch changelog.shortest from LIKE to GLOB to speed up matching · d7368933f4b0
      Jeff Sipek authored
      For whatever reason, LIKE is way slower than GLOB.  The slowdown is a
      function of the number of commits in the repository.
      
      The following data has been collected on a repository with approximately
      1.2M commits.  The numbers are in milliseconds and they represent the
      latency of the changelog.shortest function as measured by time.time().
      
      The shortest function was invoked via `hg log -l50 -T '{node|shortest}\n'`.
      
         Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
        279.0   284.3   419.8   381.7   421.3   426.2 (LIKE)
      0.08535 0.10437 0.12231 0.11919 0.12779 0.21291 (GLOB)
      d7368933f4b0
  4. Jan 09, 2025
    • Jeff Sipek's avatar
      git: add nop refresh method to gitdirstate · 482728ca94c1
      Jeff Sipek authored
      $ hg log -l1
      ...
      Traceback (most recent call last):
        File "/home/jeffpc/src/oss/hg-gitext-test/../hg-gitext/hg", line 61, in <module>
          dispatch.run()
      ...
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/dispatch.py", line 1253, in _dispatch
          repo.close()
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/localrepo.py", line 1607, in close
          self._writecaches()
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/localrepo.py", line 1611, in _writecaches
          self._revbranchcache.write()
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/branching/rev_cache.py", line 362, in write
          wlock.release()
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/lock.py", line 392, in release
          self.releasefn()
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/localrepo.py", line 3162, in unlock
          if self.dirstate.is_changing_any:
             ^^^^^^^^^^^^^
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/localrepo.py", line 225, in __get__
          return super(unfilteredpropertycache, self).__get__(unfi)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/util.py", line 1822, in __get__
          result = self.func(obj)
                   ^^^^^^^^^^^^^^
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/localrepo.py", line 1804, in dirstate
          self._dirstate.refresh()
          ^^^^^^^^^^^^^^^^^^^^^^
      AttributeError: 'gitdirstate' object has no attribute 'refresh'
      482728ca94c1
  5. Jan 04, 2025
    • Jeff Sipek's avatar
      git: mention indexing in indexing progress messages · 70b523d7a60d
      Jeff Sipek authored
      It wasn't clear what was happening to the refs and commits.  Adding
      'indexing' to the message makes it immediately obvious.
      70b523d7a60d
    • Jeff Sipek's avatar
      git: consistify & fix LookupError messages · 7d2fc79a3e5a
      Jeff Sipek authored
      This includes 3 one-line fixes:
      
      1. use the "no node" message instead of "no such node" for consistency
      2. drop superfluous (and wrong & unused) %d format string token
      3. correct one message to "no rev" since it indicates a rev that wasn't found
      7d2fc79a3e5a
  6. Oct 04, 2024
    • Jeff Sipek's avatar
      git: cache the number of commits to speed up large repo operations · 3865451a5fab
      Jeff Sipek authored
      Instead of iterating over the whole changelog table every time we want to
      know how many commits there are, we can cache the number between mercurial
      invocations.
      
      Unsurprisingly, this speeds up certain operations on repos with large
      histories.
      
      The following measurements are all in seconds and they represent the runtime
      of `hg log -T ' ' -l1 > /dev/null`.  In other words, this includes python
      startup overhead, etc.  On small and medium repos, there is no observable
      difference in runtime (because of the relatively large overhead of python
      runtime startup, and the rest of mercurial doing useful work), but on large
      repos the user-visible execution time drops by a factor of 10x or more.
      
      small repo (~600 commits):
         Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
       0.1052  0.1076  0.1096  0.1102  0.1110  0.1210 (before)
       0.1049  0.1087  0.1106  0.1120  0.1127  0.1302 (after)
      
      medium repo (12k commits):
         Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
       0.1063  0.1095  0.1116  0.1129  0.1153  0.1349 (before)
       0.1044  0.1092  0.1108  0.1115  0.1130  0.1326 (after)
      
      large repo (1.4M commits):
         Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
        1.973   2.105   2.256   2.243   2.406   2.443 (before)
        0.144   0.147   0.148   0.150   0.151   0.176 (after)
      3865451a5fab
    • Jeff Sipek's avatar
      git: index changed files on-demand · 4dadaf300fe0
      Jeff Sipek authored
      Instead of indexing the changed files for every commit immediately, we can
      index...
      
      1. heads' changed files immediately
      2. other commits' changed files on-demand
      
      This helps a lot on repositories with large histories since the initial
      mercurial invocation doesn't have to wait for the complete repo history to
      be indexed.
      4dadaf300fe0
    • Jeff Sipek's avatar
      git: track which commit's file changes have been indexed · 42f00965e50b
      Jeff Sipek authored
      Since git and mercurial commit hashes are a function of their contents, we
      can skip indexing the changed files of a commit if we have already indexed
      it as it will never change.
      
      To accomplish this, we can add a bool to the changelog table to track
      whether or not we have indexed the files of each commit.
      42f00965e50b
    • Jeff Sipek's avatar
      git: minor wording tweak to a comment · 27a0bfe770eb
      Jeff Sipek authored
      27a0bfe770eb
  7. Mar 10, 2024
    • Jeff Sipek's avatar
      git: handle octopus merges · cdbfe5e7592e
      Jeff Sipek authored
      Octopus merges in git are merge commits with more than 2 parents.  To make
      them fit into mercurial core's assumption about commits having 0-2 parents,
      the git indexing code creates "sythetic" commits to represent the octopus
      commit as a sequence of regular 2-parent commits.
      
      The synthetic commit hashes are just an incrementing commit number (which is
      the same as the generated rev number).  The last commit in the sequence of
      commits uses the actual git commit hash.  As a result, `hg checkout -r
      <commit>` produces the same working directory as `git checkout <commit>` for
      all git commit hashes.
      
      The synthetic commit hashes are stored in the changelog table as any other
      commit - with the two parents - but they also contain the commit hash of the
      octopus merge commit.
      
      For example, given the git DAG (manually pruned `git log --graph`):
      
      *-.   commit 23480d86e2689703b33f693907c40fbe6e1620e4 Merge branches...
      |\ \
      | | |
      | | * commit 2eda9984b06c75448598ec6c0a9028e49dacf616 C
      | | |
      | * | commit 5e634a12f12fedaf7b8ef0f0fcdbb07222871953 B
      | |/
      | |
      * | commit 8883a1296c5ae323a1b18d1f6410398ce43ebd3a D
      |/
      |
      * commit 95f241588fded9554cae91be0fefd576f61ebfc6 A
      
      Where M is the octopus merge commit with 3 parents, the corresponding
      mercurial DAG is:
      
      $ hg log -G -T '{node} {desc}'
      @    23480d86e2689703b33f693907c40fbe6e1620e4 Merge branches 'abc' and 'def'
      |\
      | o    0000000000000000000000000000000000000004 Merge branches 'abc' and 'def'
      | |\
      | | o  8883a1296c5ae323a1b18d1f6410398ce43ebd3a D
      | | |
      o---+  2eda9984b06c75448598ec6c0a9028e49dacf616 C
       / /
      o /  5e634a12f12fedaf7b8ef0f0fcdbb07222871953 B
      |/
      o  95f241588fded9554cae91be0fefd576f61ebfc6 A
      cdbfe5e7592e
  8. Oct 03, 2024
  9. Oct 02, 2024
    • Jeff Sipek's avatar
      git: speed up possible head processing during indexing by ~100x · aa5844ade247
      Jeff Sipek authored
      Benchmarking of 50 iterations of indexing (see below) shows that there is
      essentially no difference for small repos (<1k commits), similarly medium
      repos (~12k commits) see some benefit but other overheads completely
      overwhelm it, but for large repos (~122k commits) the 80-100x speedup is
      clearly visible to the user.
      
      All of the numbers are in seconds and were measured with time.time() calls
      placed in _index_repo().  The times exclude the time taken by changedfiles
      processing.
      
      Small repo (guilt, 553 commits, 1 head):
      
           Min.   1st Qu.    Median      Mean   3rd Qu.      Max.
      0.0008781 0.0009274 0.0009800 0.0012285 0.0014637 0.0024107 (before)
      0.0003092 0.0003281 0.0003519 0.0003777 0.0003927 0.0006843 (after)
      
      
      Medium repo (hamlib, 12k commits, 53 heads):
      
          Min.  1st Qu.   Median     Mean  3rd Qu.     Max.
      0.04881  0.05135  0.07632  0.06672  0.08042  0.09415  (before)
      0.004249 0.004420 0.004799 0.004809 0.005051 0.006416 (after)
      
      
      Large repo (qemu, 122k commits, 50 heads):
      
         Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
      4.274   4.595   4.832   6.578   8.397   9.721   (before)
      0.05180 0.05643 0.05865 0.06130 0.06712 0.06872 (after)
      aa5844ade247
    • Jeff Sipek's avatar
      git: add stub for gitdirstate.running_status · ab4fb2d15bc9
      Jeff Sipek authored
      ab4fb2d15bc9
  10. Jan 02, 2025
    • Jeff Sipek's avatar
      git: add parents property to gittreemanifestctx · 2b65c9228dc0
      Jeff Sipek authored
      2b65c9228dc0
    • Jeff Sipek's avatar
      git: add stubs for gittreemanifestctx methods · b9f47b685e90
      Jeff Sipek authored
      $ hg log
      ** unknown exception encountered, please report by visiting
      ** https://mercurial-scm.org/wiki/BugTracker
      ** Python 3.11.10 (main, Oct 31 2024, 01:10:40) [Clang 18.1.5 (https://github.com/llvm/llvm-project.git llvmorg-18.1.5-0-g617a15
      ** Mercurial Distributed SCM (version 6.9.post1.dev337+hg.72af9fa34832)
      ** Extensions loaded: absorb, churn, git (pygit2 1.14.1), githelp, gpg, hgk, histedit, patchbomb, rebase
      Traceback (most recent call last):
        File "/home/jeffpc/src/oss/hg-gitext-test/../hg-gitext/hg", line 61, in <module>
          dispatch.run()
      ...
        File "/usr/home/jeffpc/src/oss/hg-gitext/hgext/git/gitlog.py", line 564, in get
          return gitmanifest.gittreemanifestctx(self.gitrepo, t)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      TypeError: Can't instantiate abstract class gittreemanifestctx with abstract methods node, read_any_fast_delta, read_delta_new_entries, read_delta_parents, readdelta
      b9f47b685e90
    • Jeff Sipek's avatar
      git: implement baselogindex.{rev,get_rev} · 143063a94085
      Jeff Sipek authored
      143063a94085
    • Jeff Sipek's avatar
      git: add stubs for filelog methods · a26c5da900ff
      Jeff Sipek authored
      $ hg log
      ** unknown exception encountered, please report by visiting
      ** https://mercurial-scm.org/wiki/BugTracker
      ** Python 3.11.10 (main, Oct 31 2024, 01:10:40) [Clang 18.1.5 (https://github.com/llvm/llvm-project.git llvmorg-18.1.5-0-g617a15
      ** Mercurial Distributed SCM (version 6.9.post1.dev337+hg.72af9fa34832)
      ** Extensions loaded: absorb, churn, git (pygit2 1.14.1), githelp, gpg, hgk, histedit, patchbomb, rebase
      Traceback (most recent call last):
        File "/home/jeffpc/src/oss/hg-gitext-test/../hg-gitext/hg", line 61, in <module>
          dispatch.run()
      ...
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/tags.py", line 507, in _readtagcache
          if not len(repo.file(b'.hgtags')):
                     ^^^^^^^^^^^^^^^^^^^^^
        File "/usr/home/jeffpc/src/oss/hg-gitext/hgext/git/__init__.py", line 138, in file
          return gitlog.filelog(self.store.git, self.store._db, path)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      TypeError: Can't instantiate abstract class filelog with abstract methods addgroup, addrevision, censorrevision, children, commonancestorsheads, descendants, emitrevisions, files, getstrippoint, heads, iscensored, parentrevs, rawdata, revision, revs, size, storageinfo, strip, verifyintegrity
      a26c5da900ff
    • Jeff Sipek's avatar
      git: add stubs for dirstate methods · b798ad65ced8
      Jeff Sipek authored
      $ hg log
      ** unknown exception encountered, please report by visiting
      ** https://mercurial-scm.org/wiki/BugTracker
      ** Python 3.11.10 (main, Oct 31 2024, 01:10:40) [Clang 18.1.5 (https://github.com/llvm/llvm-project.git llvmorg-18.1.5-0-g617a15
      ** Mercurial Distributed SCM (version 6.9.post1.dev337+hg.72af9fa34832)
      ** Extensions loaded: absorb, churn, git (pygit2 1.14.1), githelp, gpg, hgk, histedit, patchbomb, rebase
      Traceback (most recent call last):
        File "/home/jeffpc/src/oss/hg-gitext-test/../hg-gitext/hg", line 61, in <module>
          dispatch.run()
      ...
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/localrepo.py", line 1789, in changelog
          repo.dirstate.prefetch_parents()
          ^^^^^^^^^^^^^
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/localrepo.py", line 225, in __get__
          return super(unfilteredpropertycache, self).__get__(unfi)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/util.py", line 1822, in __get__
          result = self.func(obj)
                   ^^^^^^^^^^^^^^
        File "/usr/home/jeffpc/src/oss/hg-gitext/mercurial/localrepo.py", line 1802, in dirstate
          self._dirstate = self._makedirstate()
                           ^^^^^^^^^^^^^^^^^^^^
        File "/usr/home/jeffpc/src/oss/hg-gitext/hgext/git/__init__.py", line 311, in _makedirstate
          return dirstate.gitdirstate(
                 ^^^^^^^^^^^^^^^^^^^^^
      TypeError: Can't instantiate abstract class gitdirstate with abstract methods _checkexec, _ignorefileandline, changing_files, clear, copy, hasdir, invalidate, is_changing_files, rebuild, verify
      b798ad65ced8
  11. Dec 16, 2024
    • Georges Racinet's avatar
      rust-pyo3-sharedref: demonstrate SharedByPyObject borrow unsafety · 78b2894cd58c
      Georges Racinet authored
      We take the existing doc-comment from rust-cpython and make it
      compile (validated by `cargo test`). With the added explanations,
      the ordinary comment was no longer useful, we could therefore
      remove it.
      
      The new explanation stresses that "not leaking the internal faked
      reference" is definitely not enough, because the problem is about
      *all references* that can be derived from it.
      
      We ended up duplicating the explanation, because that is a way
      to ensure that people do not miss it. Also, it was a bit misleading
      that the previous example was for `try_borrow_mut()`, so we made
      a similar, simpler one for `try_borrow()`.
      78b2894cd58c
    • Georges Racinet's avatar
      rust-pyo3-sharedref: demonstrating unsafety of map() · 76a0bdb0e4ca
      Georges Racinet authored
      We are actually about to lift it, and it will be more convincing
      if this compiling version of the old example could be refused by the
      compiler.
      
      Note: the `no_run` attribute on the example block checks that it
      compiles but does not run the example.
      76a0bdb0e4ca
  12. Dec 15, 2024
    • Georges Racinet's avatar
      rust-pyo3-sharedref: renamings in test · ac0cb3c334a1
      Georges Racinet authored
      Now that the renamings are done in main code, we give local variables
      and helper functions in the integration tests matching names.
      ac0cb3c334a1
    • Georges Racinet's avatar
      rust-pyo3-sharedref: share/map methods on PyShareable · a945845137b1
      Georges Racinet authored
      The pattern to borrow (with owner) for the sole purpose of calling
      `share_immutable`, and then `map()` is pretty common, being exactly
       what is needed in the constructors of derived Python objects.
      we therefore introduce new helpers doing exactly that. We keep the
      intermediate `share()` and `try_share()` that are used in integration
      tests, but is is possible that consumers of the API never need it.
      
      A nice feature of the new `share_map()` helper is that it collapses
      all the unsafety in constructors of derived object to one call. It makes
      sense, since the reason for unsafety is the same all along the call stack:
      the `owner` object, that cannot be guessed from `PyShareable` itself, must
      be the right one.
      
      It is remarkable that it is only at this point that the compiler insists
      that `T` should be `'static`, which is actually very reasonable, as it
      encompasses owned data. We could have set it since the beginning, but the
      added value is low, as PyO3 would not let us use some `PyShareable<&'a T>`
      as a Python object data field (with `'a` not outliving `'static` of course)
      a945845137b1
    • Georges Racinet's avatar
      rust-pyo3-sharedref: renamed UnsafePyLeaked to SharedByPyObject · 4a73eb3923ac
      Georges Racinet authored
      Rationale:
      
      - the object itself is not unsafe, only most of its methods are. This
        is similar to raw pointers: obtaining them is not unsafe, using
        them is. That being said, we are not ready yet to declare, e.g,
        `borrow_with_owner()` safe because that gives an early warning to
        users.
      - it was not really a leak, as the data is actually owned by a
        Python object on which we keep a (Python) reference with proper
        increment of the refcount. Hence the terminology was too much of
        a deterrent.
      
      We hope that the new terminology conveys both that the data
      contains a shared reference and that it was generously lended
      by a Python object.
      
      The `SharedByPyObjectRef` name is arguably heavy, but it is merely
      an implementation detail that users will not be much exposed to,
      thanks to the `Deref` implementation.
      
      As previously, we keep the changes in tests to a minimum, to make
      obvious in the diff that the tests have not changed. We will
      take care of renaming their local variables and functions in a later
      move.
      4a73eb3923ac
    • Georges Racinet's avatar
      rust-pyo3-sharedref: renamed PySharedRefCell to PyShareable · c25d345f5aa5
      Georges Racinet authored
      There were two problems with the naming:
      
      - the PyO3 version is not based on `RefCell`. Rather than calling
        this `SomethingRwLock` we decided it was best to consider this
        an implementation detail. We mentioned it in the doc earlier on,
        but that is merely just bringing some reassuring context to the
        reader, not intended to be a needed structural explanation.
      - the data is not shared: it is ready to be shared
      
      To keep the changeset readable, we only make the very minimal changes
      to the tests to make them pass, i.e., not renaming local variables
      (this will be done in a later move).
      c25d345f5aa5
    • Georges Racinet's avatar
      rust-pyo3-sharedref: replaced borrow/borrow_mut with RwLock namings · d1e304025b90
      Georges Racinet authored
      The "borrow" word had way too many uses in this context. Originally,
      it came from the rust-cpython version, which is based on `RefCell`.
      Before this change, we had untracktable stacks of borrows (first the
      `Bound`, then the `PySharedRefCell`).
      
      In any case, the change to `RwLock` is far from being neutral, and
      we may need in a future without GIL to also expose blocking methods, to
      account for in-Rust possible concurrency. Using `read()` and `write()`
      right now matches our PyO3 habits anyway, where all non-Sync objects
      have to be wrapped in a RwLock.
      d1e304025b90
    • Georges Racinet's avatar
      rust-pyo3-sharedref: reworked constructors · d85514a88706
      Georges Racinet authored
      We had previously duplicated the `new` associated function on
      `PySharedRef` with a method on `PySharedRefCell`: in `rust-cpython`,
      the former was hidden by the accessor defined by the `py_class!` macro,
      which we did not port yet.
      
      On `PySharedRefCell` itself, replacing the `new` associated function
      by the `From` trait carries all the needed semantics, and has the
      advantage of less repetititons of the type name, which will help
      with further refactorings and renamings.
      d85514a88706
  13. Dec 14, 2024
    • Georges Racinet's avatar
      rust-pyo3-sharedref: converted integration tests from rust-cpython · a7d2529ed6dd
      Georges Racinet authored
      This should bring full confidence on the conversion to PyO3.
      It highlights also the difference between the two bindings systems:
      in PyO3, the struct defined by the user is the inner Rust one.
      In rust-cpython, it is the wrapped one exposed to CPythob
      
      We enclose some of the boilerplate in helper functions.
      Perhaps we should first import the rust-cpython integration test,
      rework it with the same helpers, then only change the helpers.
      a7d2529ed6dd
  14. Dec 15, 2024
    • Georges Racinet's avatar
      rust-pyo3: initial port of PySharedRef · be765f6797cc
      Georges Racinet authored
      At this point, we have a full translation, with some tests.
      The tests are high level, demonstrating the case of iterating over
      `HashSet` as a doctest. The assertions are written as Python statements,
      in order not to be obscure and to resist later refactorings.
      The rust-cpython integration tests will be ported in a subsequent changeset.
      
      We find the example of iterating over `HashSet` to be more convincing than
      the iterating over `Vec`, the example provided in rust-cpython, because
      in the case of `Vec`, it would be simple enough to use `Arc<Vec>` and
      an index. This would of course be reimplementing the iterator, but is simple
      enough that it leads the reader to believe that having a reference is the
      problem, whereas the problem is having a reference that is itself enclosed
      in a type with lifetime that can be arbitrary complex. It took us some
      time to remember that subtlety, and hence we reworded the documentation
      to stress that point.
      
      We decided to put this work a separate crate, which makes running `cargo test`
      work for the crate, but not for the entire workspace: the `extension-module`
      feature gets in the way. That is why we reexpose the feature on `hg-pyo3` as
      default and run the tests with `--no-default-feature` in Makefile, hence in CI.
      
      An important difference with rust-cpython is that everything has to be `Sync`,
      hence we replace `RefCell` with `RwLock` and only speak of "interior mutability"
      in the documentation. Since everything happens in `hg-pyo3` behind the GIL,
      there is at this point no reason to use `read()` and `write()` instead of
      respectively `try_read()` and `try_write()`. But PyO3 is aiming to support
      free-threaded Python (without GIL), and `PySharedRef` should therefore
      allow waiting for its inner locks.
      be765f6797cc
  15. Dec 17, 2024
    • Matt Harbison's avatar
      35cc15fbc523
    • Matt Harbison's avatar
      cmdutil: switch the `mode` on `cmdutil.makefileobj()` to str · 89215c5b714c
      Matt Harbison authored
      I think the typing around whether `open()` returns `IO[bytes]` or `IO[str]`
      hinges on the content of the mode string.  Converting from bytes instead of
      using a literal can suppress that (though PyCharm currently complains about
      this).  Instead, we can mandate the use of a (vastly reduced) set of mode
      options.  For now, none of the 3 callers provide this argument, so it's not a
      big deal.  Ideally, this would always enforce binary mode.
      
      There's a little extra typing required to pull this off.  The `_unclosablefile`
      class can't subclass `typing.BinaryIO`, because there were a bunch of test
      failures around writing to stdout.  Strangely, pytype didn't complain that the
      abstract methods on `typing.BinaryIO` weren't overridden in this case.  Whatever
      was going on, it's a simple proxy class, so we can just cast to the expected
      type in the one place it is used.
      89215c5b714c
  16. Jan 01, 2025
  17. Dec 05, 2024
  18. Jan 01, 2025
    • Matt Harbison's avatar
      archival: switch the archive `kind` argument from bytes to str · 1756f5a788fd
      Matt Harbison authored
      This is intertwined with the `mode`, which is also used with `open()`, so the
      goal is to get that to str so that the `pycompat.open()` call can be dropped.
      Start simple because I had issues where the generated archive in some tests had
      1 different byte for some reason when making the full change, even though the
      content was binary equivalent when doing a Folder Compare of the archives with
      BeyondCompare4.
      
      The key for the `archivers` map is left alone for now.  The classes that are
      stored are only ever called with 2 args, so this should be safe.
      1756f5a788fd
Loading