Skip to content
Snippets Groups Projects
  1. Feb 22, 2025
  2. Feb 28, 2023
    • Matt Harbison's avatar
      revlog: add an exception hint when processing LFS flags without the extension · 10db46e128d4
      Matt Harbison authored
      It would be even better if this was either detected sooner, or the transaction
      completed (especially since the read/write processors aren't needed for the
      exchange).  But this makes it easier for the user to resolve until that can be
      figured out.
      10db46e128d4
    • Matt Harbison's avatar
      configitems: enable changegroup3 by default (unless using infinitepush) · 2cf264e9aa75
      Matt Harbison authored
      The LFS extension requires this, and if it isn't enabled on the client (or the
      LFS extension isn't loaded), a web client gets a 500 instead of a sensible error
      message.  Now it gets a different (client) error, but maybe it can be handled
      more gracefully.  c0f11347b107 indicates that treemanifest repos have this issue
      too.  29cfc474c5fd mentions gating this behind `experimental` so that the format
      could change, but that was 7 years ago and we now have an experimental
      `changegroup4` as well.
      
      We can keep this as a config for the next cycle in case someone runs into an
      unexpected problem, and then jettison it if the infinitepush bundle name changes
      are either acceptable as-is or can be created differently.
      2cf264e9aa75
  3. Feb 20, 2022
  4. Jan 07, 2022
  5. Apr 10, 2021
    • Raphaël Gomès's avatar
      cg4: introduce protocol flag to signify the presence of sidedata · 119790e1c67c
      Raphaël Gomès authored
      We need a way of signaling whether the current revision has sidedata or not,
      and re-using the revision flags would waste potential revlog flags and mix two
      normally independent layers.
      
      In this change, we add a single byte at the start of the ch4 delta header to
      set potential protocol flags. We also reclaim the revlog flag for sidedata,
      since it is no longer used, in its place now lives the (also experimental)
      copytracing flag.
      
      When generating deltas, apply the `CG_FLAG_SIDEDATA` flag if there is sidedata.
      When applying the deltas, if said flag is present, the next chunk contains the
      sidedata.
      
      Differential Revision: https://phab.mercurial-scm.org/D10343
      119790e1c67c
  6. Apr 19, 2021
  7. Mar 10, 2021
  8. Feb 15, 2021
  9. Jan 08, 2021
  10. Nov 30, 2020
  11. Nov 23, 2020
  12. Oct 01, 2020
  13. Apr 15, 2020
    • Pierre-Yves David's avatar
      upgrade: support the --quiet flag · e295ba238bd8
      Pierre-Yves David authored
      The command is currently very verbose with a various bit of output being time
      sensitive or randomized. The make invocation bulky and hard to match in the
      test. We move various message from `ui.write` to `ui.status` in order for the
      `--quiet` flag to have effect and helps the situation.
      
      As a side benefit, we can replace the various redirection to > /dev/null with the --quiet flag.
      
      Differential Revision: https://phab.mercurial-scm.org/D8429
      e295ba238bd8
    • Pierre-Yves David's avatar
      upgrade: properly filter action depending on planned work · c36a3fcfc36b
      Pierre-Yves David authored
      The `determineactions` function filters out deficiency that are not scheduled to
      be fixed by the target repository configuration. However it only did so for
      requirement we currently support, letting other actions for unsupported
      requirement through even if the target repo configuration did not request it.
      
      As a result the output of the command was easily polluted by experimental
      feature with no upgrade support.
      
      We rework the code to still filter out requirement based action without the
      faulty filtering.
      
      Differential Revision: https://phab.mercurial-scm.org/D8427
      c36a3fcfc36b
  14. Jan 21, 2020
    • Matt Harbison's avatar
      lfs: enable workers by default · 87167caa2989
      Matt Harbison authored
      With the stall issue seemingly fixed, there's no reason not to use workers.  The
      setting is left for now to keep the test output deterministic, and in case other
      issues come up.  If none do, this can be converted to a developer setting for
      usage with the tests.
      
      Differential Revision: https://phab.mercurial-scm.org/D7963
      87167caa2989
  15. Oct 07, 2019
  16. Sep 04, 2019
  17. Sep 08, 2019
    • Pierre-Yves David's avatar
      changegroup: move message about added changes to transaction summary · d7304434390f
      Pierre-Yves David authored
      Before that, applying multiple changegroups in the same transaction issued the
      message multiple time. This result in a confusing output:
      
          adding changesets
          adding manifests
          adding file changes
          added 32768 changesets with 60829 changes to 2668 files
          adding changesets
          adding manifests
          adding file changes
          added 8192 changesets with 16885 changes to 1553 files
          adding changesets
          adding manifests
          adding file changes
          added 1020 changesets with 1799 changes to 536 files
          adding changesets
          adding manifests
          ...
      
      Instead, we now only issue the message once at the end of the transaction,
      summing up all added changesets, changes and files. The line is identical, but
      happens sightly later in the output.
      
      There are other suboptimal behavior around issue multiple changegroup (eg:
      progress bar). We'll cover them later.
      
      This impact of lot of test as one would expect, but a two pass check show they
      are just the order change we expected.
      
      To deal with "under the hood" bundle application by internal code, we had to
      take a slightly hacky move. We could clean that up with a more official way to
      enter "under the hood" section, however I want to keep this series simple to get
      it landed. This kind of change have a very high bit rot rate since it impact a
      lot of test output.
      d7304434390f
  18. Oct 04, 2018
    • Matt Harbison's avatar
      cat: don't prefetch files unless the output requires it · 561cd02c58ff
      Matt Harbison authored
      It's a waste to cache lfs blobs when cat'ing the raw data at best, but a hassle
      debugging when the blob is missing.  I'm not sure if there are other commands
      that have '{data}' for output, and if there's a general way to prefetch on that
      keyword.
      
      It's interesting that the verbose output seems to leak into the JSON output, but
      that seems like an existing bug.
      561cd02c58ff
  19. Feb 15, 2019
  20. Jan 28, 2019
  21. Jan 27, 2019
  22. Jan 18, 2019
  23. Nov 12, 2018
  24. Sep 21, 2018
    • Matt Harbison's avatar
      lfs: autoload the extension when cloning from repo with lfs enabled · 6637b079ae45
      Matt Harbison authored
      This is based on a patch by Gregory Szorc.  I made small adjustments to
      clean up the messaging when the server has the extension enabled, but the
      client has it disabled (to prevent autoloading).  Additionally, I added
      a second server capability to distinguish between the server having the
      extension enabled, and the server having LFS commits.  This helps prevent
      unnecessary requirement propagation- the client shouldn't add a requirement
      that the server doesn't have, just because the server had the extension
      loaded.  The TODO I had about advertising a capability when the server can
      natively serve up blobs isn't relevant anymore (we've had 2 releases that
      support this), so I dropped it.
      
      Currently, we lazily add the "lfs" requirement to a repo when we first
      encounter LFS data. Due to a pretxnchangegroup hook that looks for LFS
      data, this can happen at the end of clone.
      
      Now that we have more control over how repositories are created, we can
      do better.
      
      This commit adds a repo creation option to add the "lfs" requirement.
      hg.clone() sets this creation option if the remote peer is advertising
      lfs usage (as opposed to just support needed to push).
      
      So, what this change effectively does is have cloned repos
      automatically inherit the "lfs" requirement.
      
      Differential Revision: https://phab.mercurial-scm.org/D5130
      6637b079ae45
  25. Oct 16, 2018
  26. Oct 04, 2018
    • Matt Harbison's avatar
      lfs: register the flag processors per repository · 9c4cbbb0fc51
      Matt Harbison authored
      Previously, enabling the extension for any repo in commandserver or hgweb would
      enable the flags on all repos.  Since localrepo.resolverevlogstorevfsoptions()
      is called so early, the check to see if the extension is enabled on the repo
      (which hasn't been instantiated yet) is a bit awkward.  But I don't see a better
      way.
      9c4cbbb0fc51
  27. Oct 10, 2018
    • Matt Harbison's avatar
      lfs: avoid a potential variable reference before assignment error in cmdserver · 535fc8a22365
      Matt Harbison authored
      A coworker hit this once yesterday when pulling in thg (a retry worked), and
      then I hit it with strip after a pull.  I had a difficult time recreating a test
      for this (at least one of the tricks was to not use '-R', which seems to cause
      reposetup() to be called for each command), so I'm not sure how large of a
      window there actually is for this.  Calling reposetup() *after* the requirement
      is added will skip the hook entirely.
      
      The other issue I had was adding a couple `ui.status()` lines around the check
      that installs the hook.  On Windows, the cmdserver process ballooned to 1.6GB
      and hung.  Changing that to `ui.warn()` avoided the hang.  It also hung on
      macOS, but without the large memory usage.
      535fc8a22365
  28. Oct 01, 2018
    • Gregory Szorc's avatar
      httppeer: report http statistics · 393e44324037
      Gregory Szorc authored
      Now that keepalive.py records HTTP request count and the
      number of bytes sent and received as part of performing those
      requests, we can easily print a report on the activity when
      closing a peer instance!
      
      Exact byte counts are globbed in tests because they are influenced
      by non-deterministic things, such as hostnames and port numbers.
      Plus, the exact byte count isn't too important anyway.
      
      I feel obliged to note that printing the byte count could have
      security implications. e.g. if sending a password via HTTP basic
      auth, the length of that password will influence the byte count
      and the reporting of the byte count could be a side-channel leak
      of the password length. I /think/ this is beyond our threshold
      for concern. But if we think it poses a problem, we can teach the
      byte count logging code to e.g. ignore sensitive HTTP request
      headers. We could also consider not reporting the byte count of
      request headers altogether. But since the wire protocol uses HTTP
      headers for sending command arguments, it is kind of important to
      report their size.
      
      Differential Revision: https://phab.mercurial-scm.org/D4858
      393e44324037
  29. Sep 24, 2018
    • Gregory Szorc's avatar
      upgrade: report size of backing files, not internal storage size · 8dab7c8a93eb
      Gregory Szorc authored
      upgrade.py is the only consumer of filelog.index, which I'd like
      to eliminate from the file storage interface.
      
      This commit changes the upgrade code to report the storage size
      of files by looking at the size of the files backing its storage
      instead of looking at the index.
      
      I'm not convinced the approach in this patch will live very long
      because it is relying on low-level attributes like "opener" and
      "files," which may behave very differently on non-revlog storage.
      But the data is only used for reporting purposes and it does get
      us one step closer to eliminating "index."
      
      A side-effect of this change is we now report the size of the revlog
      index data - not just the revision data. I think this is more
      accurate.
      
      Differential Revision: https://phab.mercurial-scm.org/D4717
      8dab7c8a93eb
  30. Sep 19, 2018
  31. Jun 06, 2018
  32. May 31, 2018
    • Matt Harbison's avatar
      lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902) · 3790efb388ca
      Matt Harbison authored
      There are only a handful of methods that access repo attributes that are applied
      in reposetup().  The `diff` test covers all of the commands that call
      scmutil.prefetchfiles().  Along the way, I saw that adding files and upgrading
      the repo format were also problems (also tested here).
      
      I don't think running `hg serve` through the commandserver is sane, but I
      conditionalized both the capabilities and the wsgirequest handler because it's
      trivially correct.  It doesn't look like there has ever been a caller of
      candownload(), so there's no test for that path.
      
      The upload case isn't testable, because uploadblobs() bails if there are no
      pointers.  The requirement should be added any time pointers are introduced, and
      that would force the extension to be loaded specifically for the repo.  This
      covers `debuglfsupload`, the pre-push hook (which isn't set until the repo is
      promoted to LFS), and uploadblobsfromrevs(), which can be called by other
      extensions.
      
      I think readfromstore() and writetostore() are only reachable as a flag
      processor for revlog.REVIDX_EXTSTORED, and a requirement is added as soon as
      that is seen, so I don't think those are a problem.
      3790efb388ca
  33. May 15, 2018
  34. Apr 15, 2018
    • Matt Harbison's avatar
      lfs: enable the final download count status message · ab04972a33ef
      Matt Harbison authored
      At this point, I think all of the core commands are prefetching, except grep and
      verify.  Verify will need some special handling, in case the revlogs are
      corrupt.
      
      Grep has an issue that still needs to be debugged, but we probably need to give
      the behavior some thought too- it would be a shame to have to download
      everything in order to search.  I think the benefit of having this info for all
      commands outweighs extra printing in a command that is arguably not well
      behaved in this context anyway.
      ab04972a33ef
    • Matt Harbison's avatar
      diff: invoke the file prefetch hook · 5e67c20915a7
      Matt Harbison authored
      By invoking it this deep within the command, we pick up both subrepo and hgweb
      support, as well as --patch support for commands that implement logopts.
      5e67c20915a7
Loading