Skip to content
Snippets Groups Projects
  1. Apr 10, 2015
    • Martin von Zweigbergk's avatar
      treemanifest: lazily load manifests · 0de132d5328a
      Martin von Zweigbergk authored
      Most operations on treemanifests already visit only relevant
      submanifests. Notable examples include __getitem__, __contains__,
      walk/matches with matcher, diff. By making submanifests lazily loaded,
      we speed up all these operations.
      
      The lazy loading is achieved by adding a _load() method that gets
      defined where we currently eagerly parse the manifest. We make sure to
      call it before any access to _dirs, _files or _flags.
      
      Some timings on the Mozilla repo (with flat manifest timings for
      reference):
      
      hg cat -r . README.txt: 1.644s -> 0.096s (0.255s)
      hg diff -r .^ -r .    : 1.746s -> 0.137s (0.431s)
      hg files -r . python  : 1.508s -> 0.146s (0.335s)
      hg files -r .         : 2.125s -> 2.203s (0.712s)
      0de132d5328a
  2. May 19, 2015
    • Martin von Zweigbergk's avatar
      treemanifest: speed up commit using dirty flag · eafa06e9edde
      Martin von Zweigbergk authored
      We currently avoid saving a treemanifest revision if it's the same as
      one of it's parents. This is checked by comparing the generated text
      for all three versions. Let's avoid that when possible by comparing
      the nodeids for clean (not dirty) nodes.
      
      On the Mozilla repo, this speeds up commit from 2.836s to 2.343s.
      eafa06e9edde
  3. Feb 26, 2015
    • Martin von Zweigbergk's avatar
      treemanifest: speed up diff by keeping track of dirty nodes · f0fbd88b21fb
      Martin von Zweigbergk authored
      Since tree manifests have a nodeid per directory, we can avoid diffing
      entire directories if they have the same nodeid. The comparison is
      only valid for unmodified treemanifest instances, of course, so we
      need to keep track of which have been modified. Therefore, let's add a
      dirty flag to treemanifest indicating whether its nodeid can be
      trusted. We set it when _files or _dirs is modified, and make diff(),
      and its cousin filesnotin(), not descend into subdirectories that are
      the same on both sides.
      
      On the Mozilla repo, this speeds up 'hg diff -r .^ -r .' from 1.990s
      to 1.762s. The improvement will be much larger when we start lazily
      loading subdirectory manifests.
      f0fbd88b21fb
  4. Nov 05, 2014
    • Martin von Zweigbergk's avatar
      test-walk: add more tests for -I/-X · 70e822796ac8
      Martin von Zweigbergk authored
      We had very limited testing of -I and -X, especially when combined
      with plain file patterns and with each other. This adds some more
      protection against regressions as upcoming patches modify the matcher
      code. (Originally meant for my own upcoming patches, but now I know
      drgott will be sending some patches soon.)
      
      The only noteworthy cases seems to be that both of
      
        hg debugwalk -Xbeans/black beans/black
        hg debugwalk -Xbeans beans/black
      
      walk the file. I would personally have expected the -X to trump. I
      don't care enough to change it, but I also think it's fair if some
      future commit changes the behavior.
      70e822796ac8
  5. May 16, 2015
    • Durham Goode's avatar
      ignore: use 'include:' rules instead of custom syntax · dc562165044a
      Durham Goode authored
      Now that the matcher supports 'include:' rules, let's change the dirstate.ignore
      creation to just create a matcher with a bunch of includes. This allows us to
      completely delete ignore.py.
      
      I moved some of the syntax documentation over to readpatternfile in match.py so
      we don't lose it.
      dc562165044a
    • Durham Goode's avatar
      match: add 'include:' syntax · 4040e06e9b99
      Durham Goode authored
      This allows the matcher to understand 'include:path/to/file' style rules.  The
      files support the standard hgignore syntax and any rules read from the file are
      included in the matcher without regard to the files location in the repository
      (i.e. if the included file is in somedir/otherdir, all of it's rules will still
      apply to the entire repository).
      4040e06e9b99
  6. May 18, 2015
    • Durham Goode's avatar
      match: add optional warn argument · 08703b10c3ae
      Durham Goode authored
      Occasionally the matcher will want to print warning messages instead of throwing
      exceptions (like if it encounters a bad syntax parameter when parsing files).
      Let's add an optional warn argument that can provide this. The next patch will
      actually use this argument.
      08703b10c3ae
  7. May 16, 2015
    • Durham Goode's avatar
      match: add source to kindpats list · 08a8e9da0ae7
      Durham Goode authored
      Future patches will be adding the ability to recursively include pattern files
      in a match rule expression. Part of that behavior will require tracking which
      file each pattern came from so we can report errors correctly.
      
      Let's add a 'source' arg to the kindpats list to track this. Initially it will
      only be populated by listfile rules.
      08a8e9da0ae7
  8. May 19, 2015
  9. May 18, 2015
  10. May 19, 2015
  11. May 18, 2015
    • Matt Harbison's avatar
      match: explicitly naming a subrepo implies always() for the submatcher · ef4538ba67ef
      Matt Harbison authored
      The files command supports naming directories to limit the output to children of
      that directory, and it also supports -S to force recursion into a subrepo.  But
      previously, using -S and naming a subrepo caused nothing to be output.  The
      reason was narrowmatcher() strips the current subrepo path off of each file,
      which would leave an empty list if only the subrepo was named.
      
      When matching on workingctx, dirstate.matches() would see the submatcher is not
      always(), so it returned the list of files in dmap for each file in the matcher-
      namely, an empty list.  If a directory in a subrepo was named, the output was as
      expected, so this was inconsistent.
      
      The 'not anypats()' check is enforced by an existing test around line 140:
      
          $ hg remove -I 're:.*.txt' sub1
      
      Without the check, this removed all of the files in the subrepo.
      ef4538ba67ef
  12. May 17, 2015
    • Matt Harbison's avatar
      context: don't complain about a matcher's subrepo paths in changectx.walk() · ccb1623266eb
      Matt Harbison authored
      Previously, the first added test printed the following:
      
        $ hg files -S -r '.^' sub1/sub2/folder
        sub1/sub2/folder: no such file in rev 9bb10eebee29
        sub1/sub2/folder: no such file in rev 9bb10eebee29
        sub1/sub2/folder/test.txt
      
      One warning occured each time a subrepo was crossed into.
      
      The second test ensures that the matcher copy stays in place.  Without the copy,
      the bad() function becomes an increasingly longer chain, and no message would be
      printed out for a file missing in the subrepo because the predicate would match
      in one of the replaced methods.  Manifest doesn't know anything about subrepos,
      so it needs help ignoring subrepos when complaining about bad files.
      ccb1623266eb
  13. May 19, 2015
    • Pierre-Yves David's avatar
      ssh: capture output with bundle2 again (issue4642) · 36111f98f23d
      Pierre-Yves David authored
      I just discovered that we are not displaying ssh server output in real time
      anymore. So we can just fall back to the bundle2 output capture for now. This
      fix the race condition issue we where seeing in tests. Re-instating real time
      output for ssh would fix the issue too but lets get the test to pass first.
      36111f98f23d
  14. Apr 24, 2015
    • Laurent Charignon's avatar
      revset: optimize not public revset · 08d1ef09ed37
      Laurent Charignon authored
      This patvh speeds up the computation of the not public() changeset
      and incidentally speed up the computation of divergents() changeset on our big
      repo by 100x from 50% to 0.5% of the time spent in smartlog with evolve.
      
      In this patch we optimize not public() to _notpublic() (new revset) and use
      the work on phaseset (from the previous commit) to be able to compute
      _notpublic() quickly.
      
      We use a non-lazy approach making the assumption the number of notpublic
      change will not be in the order of magnitude of the repo size. Adopting a
      lazy approach gives a speedup of 5x (vs 100x) only due to the overhead of the
      code for lazy generation.
      08d1ef09ed37
  15. Apr 01, 2015
    • Laurent Charignon's avatar
      phases: add set per phase in C phase computation · 22438cfd11b5
      Laurent Charignon authored
      To speed up the computation of draft(), secret(), divergent(), obsolete() and
      unstable() we need to have a fast way of getting the list of revisions that
      are in draft(), secret() or the union of both: not public().
      
      This patch extends the work on phase computation in C and make the phase
      computation code also return a list of set for each non public phase.
      Using these sets we can quickly obtain all the revisions of a given phase.
      We do not return a set for the public phase as we expect it to be roughly the
      size of the repo. Also, it can be computed easily by substracting the entries in the
      non public phases from all the revs in the repo.
      22438cfd11b5
  16. May 08, 2015
  17. May 06, 2015
    • Drew Gottlieb's avatar
      match: remove unnecessary optimization where visitdir() returns 'all' · 2773540c3650
      Drew Gottlieb authored
      Match's visitdir() was prematurely optimized to return 'all' in some cases, so
      that the caller would not have to call it for directories within the current
      directory. This change makes the visitdir system less flexible for future
      changes, such as making visitdir consider the match's include and exclude
      patterns.
      
      As a demonstration of this optimization not actually improving performance,
      I ran 'hg files -r . media' on the Mozilla repository, stored as treemanifest
      revlogs.
      
      With best of ten tries, the command took 1.07s both with and without the
      optimization, even though the optimization reduced the calls from visitdir()
      from 987 to 51.
      2773540c3650
  18. Apr 16, 2015
  19. Apr 28, 2015
    • Augie Fackler's avatar
      extensions: document that `testedwith = 'internal'` is special · 80c5b2666a96
      Augie Fackler authored
      Extension authors (notably at companies using hg) have been
      cargo-culting the `testedwith = 'internal'` bit from hg's own
      extensions, which then defeats our "file bugs over here" logic in
      dispatch. Let's be more aggressive about trying to give extension
      authors a hint about what testedwith should say.
      80c5b2666a96
  20. Apr 11, 2015
    • Martin von Zweigbergk's avatar
      treemanifest: cache directory logs and manifests · bf6b476f3b36
      Martin von Zweigbergk authored
      Since manifests instances are cached on the manifest log instance, we
      can cache directory manifests by caching the directory manifest
      logs. The directory manifest log cache is a plain dict, so it never
      expires; we assume that we can keep all the directories in memory.
      
      The cache is kept on the root manifestlog, so access to directory
      manifest logs now has to go through the root manifest log.
      
      The caching will soon not be only an optimization. When we start
      lazily loading directory manifests, we need to make sure we don't
      create multiple instances of the log objects. The caching takes care
      of that problem.
      bf6b476f3b36
  21. May 18, 2015
Loading