Skip to content
Snippets Groups Projects
  1. Dec 14, 2015
  2. Dec 17, 2015
  3. Dec 07, 2015
    • Matt Mackall's avatar
      run-tests: show scheduling with --showchannels · 64c58407
      Matt Mackall authored
      This gives one line of output per second with one column per -j level
      that allows analyzing test scheduling problems. First 24 seconds of
      output at -j 30 looks like this:
      
        0                                                                .
        1  = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =   s.
        2  c c o c r l g r s s = c p = c h c a h c g c h c b c c l l c   ss
        3  h o b o e a e u u u c o a h o e o c g o l h g h u o = a o =   s
        4  e n s n b r n n b b   m t g n l n l w n o e w e n n e r g i   .
        5  c t o = a g d - r r = m c w v p v . e v g c e c d v x g . m
        6  k r l r s e o t e e b a h e e . e . b e . k b k l e t e . p
        7  - i e e e f c e p p u n b b r . r . - r . - - - e r e f . o   .
        8  p b t v - i . s o o n d o d t . t . c t . c s = 2 t n i . r
        9  y - e s c l . t - . d - m i - . - . o - . o y r - - s l . t
       10  3 p - e h e . s s . l t b r s . s . m s . d m e f s i e . .
       11  - e c t e s . . v . e e . . v . v . m v . e r n o v o s . .
       12  c r h . c - . . n . 2 m . . n . n . a n . . e a r n n . . .
       13  o f e . k u . . . . - p . . - . - . n - . . v m m - . . . .
       14  m . c . - p . . . . e l . . s . m . d s . . . e a e . . . .
       15  p . k . r d . . . . x a . . i . o . s o . . . - t n . . . .
       16  a . h . e a . . . . c t . . n . v . . u . . . m . c . . . .
       17  t . e . s t . . . . h e . . k . e . . r . . . e . o . . . .
       18  . . a . t e . . . . a . . . . . . . . c . . . r . d . . . .
       19  . . d . o . . . . . n . . . . . . . . e . . . g . i . . . .
       20  . . s . r . . . . . g . . . . . . . . . . . . e . n . . . .
       21  . . . . e . . . . . e . . . . . . . . . . . . 2 . g . . . .
       22  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
       23  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   .
       24  . . . . . . . . . . . . . . . . . . . . . . . . . = . . . .   ^C
      
      Test names read off vertically, beginning with '='. Idle time (not
      shown) appears as blank space.
      64c58407
  4. Dec 06, 2015
  5. Dec 04, 2015
    • Matt Mackall's avatar
      run-tests: add more scheduling weight hints · b4d7743e
      Matt Mackall authored
      The scheduler would like to order test execution by expected run-time,
      but doesn't know much about how long a test will run. It thus uses
      test size as a proxy for run-time. By tweaking these weights we can
      keep CPUs more evenly busy and thus finish sooner.
      
      In particular, this change pushes the three currently longest-running
      tests closer to the beginning:
      
      test-largefiles-update.t
      test-run-tests.t
      test-gendoc.t
      
      As the largefiles test is currently the long pole of the test suite
      with higher -j factors, the sooner it's started, the sooner the tests
      can end.
      
      We also up the weight on some shorter but long-running tests that
      could have previously delayed completion with low -j factors by
      running very close to the end.
      b4d7743e
    • Matt Mackall's avatar
      run-tests: report timeouts in a less alarming fashion · a40b623e
      Matt Mackall authored
      Rather than report timed-out tests like this:
      
       ERROR: test-convert-svn-sink.t output changed
       !
      
      ..simply put a 't' rather than a '.' in the stream.
      a40b623e
  6. Dec 16, 2015
  7. Dec 15, 2015
  8. Dec 14, 2015
  9. Dec 16, 2015
  10. Sep 14, 2015
    • Matt Harbison's avatar
      tests: add coverage to ensure Wix tracks 'help' and 'templates' files · b1160299
      Matt Harbison authored
      This would have caught the problem fixed by 65d2538ac993. There are
      other *.wxs files that can be checked, but they appear to be more
      complicated. For example, locale.wxs has what appears to be foreach
      loop support, as well as variable substitution.
      
      By checking `hg files` to determine tracked file, this is able to avoid false
      failures when other junk is present in the filesystem, like *.orig files.
      
      I can't tell if the map-cmdline.status file is not included on purpose, but I
      don't see the purpose of excluding it.  The missing help files seem reasonable
      for Windows.
      b1160299
  11. Dec 13, 2015
  12. Oct 24, 2015
    • Gregory Szorc's avatar
      help: add documentation for changegroup formats · a79cba6c
      Gregory Szorc authored
      There is no formal location for spec-like technical/internal docs. The
      repository makes sense as such a location because spec-like
      documentation should be reviewed (ruling out a wiki). mpm has also
      stated that he would like this documentation to be part of the
      built-in help system. So, we establish an "internals" sub-directory
      to hold this class of documentation.
      
      The format of changegroups does not appear to be documented anywhere,
      even in source code. It therefore seemed like an appropriate first thing
      to document.
      
      This patch adds low-level documentation of versions 1 and 2 of the
      changegroup foromat. It currently only describes the raw data format.
      There is probably room to write higher-level documentation on strategies
      for producing and consuming the data. We'll leave that for another day.
      
      The added file is not yet accessible via `hg help` nor via hgweb.
      Support for this will follow in subsequent patches.
      a79cba6c
  13. Dec 07, 2015
    • Gregory Szorc's avatar
      util: reimplement lrucachedict · 45d996a5
      Gregory Szorc authored
      As part of attempting to more aggressively use the existing
      lrucachedict, collections.deque operations were frequently
      showing up in profiling output, negating benefits of caching.
      
      Searching the internet seems to tell me that the most efficient
      way to implement an LRU cache in Python is to have a dict indexing
      the cached entries and then to use a doubly linked list to track
      freshness of each entry. So, this patch replaces our existing
      lrucachedict with a version using such a pattern.
      
      The recently introduced perflrucachedict command reveals the
      following timings for 10,000 operations for the following cache
      sizes for the existing cache:
      
      n=4     init=0.004079 gets=0.003632 sets=0.005188 mixed=0.005402
      n=8     init=0.004045 gets=0.003998 sets=0.005064 mixed=0.005328
      n=16    init=0.004011 gets=0.004496 sets=0.005021 mixed=0.005555
      n=32    init=0.004064 gets=0.005611 sets=0.005188 mixed=0.006189
      n=64    init=0.003975 gets=0.007684 sets=0.005178 mixed=0.007245
      n=128   init=0.004121 gets=0.012005 sets=0.005422 mixed=0.009471
      n=256   init=0.004143 gets=0.020295 sets=0.005227 mixed=0.013612
      n=512   init=0.004039 gets=0.036703 sets=0.005243 mixed=0.020685
      n=1024  init=0.004193 gets=0.068142 sets=0.005251 mixed=0.033064
      n=2048  init=0.004070 gets=0.133383 sets=0.005160 mixed=0.050359
      n=4096  init=0.004053 gets=0.265194 sets=0.004868 mixed=0.048352
      n=8192  init=0.004087 gets=0.542218 sets=0.004562 mixed=0.032753
      n=16384 init=0.004106 gets=1.064055 sets=0.004179 mixed=0.020367
      n=32768 init=0.004034 gets=2.097620 sets=0.004260 mixed=0.013031
      n=65536 init=0.004108 gets=4.106390 sets=0.004268 mixed=0.010191
      
      As the data shows, the existing cache's retrieval performance
      diminishes linearly with cache size. (Keep in mind the microbenchmark
      is testing 100% cache hit rate.)
      
      The new cache implementation reveals the following:
      
      n=4     init=0.006665 gets=0.006541 sets=0.005733 mixed=0.006876
      n=8     init=0.006649 gets=0.006374 sets=0.005663 mixed=0.006899
      n=16    init=0.006570 gets=0.006504 sets=0.005799 mixed=0.007057
      n=32    init=0.006854 gets=0.006459 sets=0.005747 mixed=0.007034
      n=64    init=0.006580 gets=0.006495 sets=0.005740 mixed=0.006992
      n=128   init=0.006534 gets=0.006739 sets=0.005648 mixed=0.007124
      n=256   init=0.006669 gets=0.006773 sets=0.005824 mixed=0.007151
      n=512   init=0.006701 gets=0.007061 sets=0.006042 mixed=0.007372
      n=1024  init=0.006641 gets=0.007620 sets=0.006387 mixed=0.007464
      n=2048  init=0.006517 gets=0.008598 sets=0.006871 mixed=0.008077
      n=4096  init=0.006720 gets=0.010933 sets=0.007854 mixed=0.008663
      n=8192  init=0.007383 gets=0.015969 sets=0.010288 mixed=0.008896
      n=16384 init=0.006660 gets=0.025447 sets=0.011208 mixed=0.008826
      n=32768 init=0.006658 gets=0.044390 sets=0.011192 mixed=0.008943
      n=65536 init=0.006836 gets=0.082736 sets=0.011151 mixed=0.008826
      
      Let's go through the results.
      
      The new cache takes longer to construct. ~6.6ms vs ~4.1ms. However,
      this is measuring 10,000 __init__ calls, so the difference is
      ~0.2us/instance. We currently only create lrucachedict for manifest
      instances, so this regression is not likely relevant.
      
      The new cache is slightly slower for retrievals for cache sizes
      < 1024. It's worth noting that the only existing use of lurcachedict
      is in manifest.py and the default cache size is 4. This regression
      is worrisome. However, for n=4, the delta is ~2.9s for 10,000 lookups,
      or ~0.29us/op. Again, this is a marginal regression and likely not
      relevant in the real world. Timing `hg log -p -l 100` for
      mozilla-central reveals that cache lookup times are dominated by
      decompression and fulltext resolution (even with lz4 manifests).
      
      The new cache is significantly faster for retrievals at larger
      capacities. Whereas the old implementation has retrieval performance
      linear with cache capacity, the new cache is constant time until much
      larger values. And, when it does start to increase significantly, it
      is a few magnitudes faster than the current cache.
      
      The new cache does appear to be slower for sets when capacity is large.
      However, performance is similar for smaller capacities. Of course,
      caches should generally be optimized for retrieval performance because
      if a cache is getting more sets than gets, it doesn't really make
      sense to cache. If this regression is worrisome, again, taking the
      largest regression at n=65536 of ~6.9ms for 10,000 results in a
      regression of ~0.68us/op. This is not significant in the grand scheme
      of things.
      
      Overall, the new cache is performant at retrievals at much larger
      capacity values which makes it a generally more useful cache backend.
      While there are regressions, their absolute value is extremely small.
      Since we aren't using lrucachedict aggressively today, these
      regressions should not be relevant. The improved scalability of
      lrucachedict should enable us to more aggressively utilize
      lrucachedict for more granular caching (read: higher capacity caches)
      in the near future. The impetus for this patch is to establish a cache
      of decompressed revlog revisions, notably manifest revisions. And since
      delta chains can grow to >10,000 and cache hit rate can be high, the
      improved retrieval performance of lrucachedict should be relevant.
      45d996a5
  14. Dec 12, 2015
    • Siddharth Agarwal's avatar
      record: don't dereference symlinks while copying over stat data · d9e3ebe5
      Siddharth Agarwal authored
      Previously, we could be calling os.utime or os.chflags (via shutil.copystat) on
      a symlink. These functions dereference symlinks, so this would have caused the
      timestamp of the target to be set. On a read-only or similarly weird
      filesystem, this might cause an exception to be raised.
      
      This is pretty hard to test because conjuring up a read-only filesystem for
      test purposes is non-trivial.
      d9e3ebe5
    • Siddharth Agarwal's avatar
      copyfile: add an optional parameter to copy other stat data · c48ecc0b
      Siddharth Agarwal authored
      Contrary to the comment, I didn't see any evidence that we were copying
      atime/mtime at all. This adds a parameter to copyfile to optionally copy it and
      other stat data, with the default being to not copy it.
      
      Many systems don't support changing the timestamp of a symlink, but we don't
      need that in general anyway -- copystat is mostly useful for editors, most of
      which will dereference symlinks anyway.
      c48ecc0b
  15. Dec 06, 2015
    • Pierre-Yves David's avatar
      tests: move the '-hg' postfix for all style tests · 409a2031
      Pierre-Yves David authored
      We had them on 'test-check-code-hg.t' to avoid collision with the test checking
      'check-code' itself. Now that this one have been rename, we can safely remove
      this suffix for all of them. This get them in line with 'check-pyflakes.t'.
      409a2031
    • Pierre-Yves David's avatar
      test: rename 'check-code' own test to 'test-contrib-check-code.t' · 8833dadd
      Pierre-Yves David authored
      This test (making sure the 'check-code' script run as intended) have been
      confused with the test making that the mercurial code base comply with our
      coding still by multiple generations of contributors.
      
      We are moving it out of the way so that all tests starting with
      'test-check' are now doing compliance testing.
      8833dadd
  16. Dec 14, 2015
Loading