Skip to content
Snippets Groups Projects
  1. Oct 06, 2015
  2. Oct 04, 2015
    • Yuya Nishihara's avatar
      util: use tuple accessor to get accurate st_mtime value (issue4836) · 13272104
      Yuya Nishihara authored
      Because st.st_mtime is computed as 'sec + 1e-9 * nsec' and double is too narrow
      to represent nanoseconds, int(st.st_mtime) can be 'sec + 1'. Therefore, that
      value could be different from the one got by osutils.listdir().
      
      This patch fixes the problem by accessing to raw st_mtime by tuple index.
      
      It catches TypeError to fall back to st.st_mtime because our osutil.stat does
      not support tuple index. In dirstate.normal(), 'st' is always a Python stat,
      but in dirstate.status(), it can be either a Python stat or an osutil.stat.
      
      Thanks to vgatien-baron@janestreet.com for finding the root cause of this
      subtle problem.
      13272104
    • Yuya Nishihara's avatar
      util: extract stub function to get mtime with second accuracy · 3a0bb613
      Yuya Nishihara authored
      This function is trivial but will need a long comment why it can't use
      st.st_mtime. See the next patch for details.
      3a0bb613
  3. Oct 05, 2015
  4. Sep 24, 2015
  5. Aug 25, 2015
  6. Aug 24, 2015
    • Matt Harbison's avatar
      templater: introduce {latesttag()} function to match a pattern (issue4184) · 43bf9471
      Matt Harbison authored
      This allows the latest class of tag to be found, such as a release candidate or
      final build, instead of just the absolute latest.
      
      It doesn't appear that the existing keyword can be given an optional argument.
      There is a keyword, function and filter for 'date', so it doesn't seem harmful
      to introduce a new function with the same name as an existing keyword.  Most
      functions are pretty Mercurial agnostic, but there is {revset()} as precedent.
      
      Even though templatekw.getlatesttags() returns a single tuple, one entry of
      which is a list, it is simplest to present this as a list of tags instead of a
      single item, with each tag having a distance and change count attribute.  It is
      also closer to how {latesttag} returns a list of tags, and how this function
      works when not given a '%' operator.
      43bf9471
  7. Aug 25, 2015
    • Matt Harbison's avatar
      templatekw: introduce showlatesttags() to handle {latesttag} keywords · 93c80e7e
      Matt Harbison authored
      The keywords {changes}, {distance} and {tag} will be available on a future
      template method that will allow pattern matching against tag names.  For
      consistency, these should be available on the existing {latesttag} keyword as
      well.
      
      I debated whether or not to add {tag} instead of just continuing with the
      existing {latesttag}.  But it seems clearer not to have the same name for two
      distinct things (a list in the LHS of %, and an individual tag value on the
      right).
      
      The value of latesttags[0] is the date of commit for the cset to which the tag
      is applied (i.e. not the date the tag was applied), and therefore isn't made
      visible because it doesn't seem interesting.  It appears that this is merely an
      internal implementation detail for sorting csets in a stable manner when there
      are different branches.
      93c80e7e
  8. Oct 06, 2015
  9. Aug 24, 2015
    • Matt Harbison's avatar
      templatekw: allow getlatesttags() to match a specific tag pattern · d2e69584
      Matt Harbison authored
      This will allow the latest class of tag to be found, such as a release candidate
      or final build, instead of just the absolute latest.  It will be exposed in a
      future patch.
      
      It's unfortunate that the original 'latesttags' cache can't be used to determine
      the proper values, but it isn't fully populated for the entire repo.  For
      example, the {latesttagdistance} keyword on the Mecurial repo builds the cache
      up back to the revision for 1.4.  If the pattern was 're:^0\.\d$', that wouldn't
      be in the cache.  Maybe this can be optimized some other way, but for now, this
      is the simpliest implementation.
      d2e69584
  10. Aug 23, 2015
  11. Oct 06, 2015
    • Gregory Szorc's avatar
      util.chunkbuffer: avoid extra mutations when reading partial chunks · 6ae14d1c
      Gregory Szorc authored
      Previously, a read(N) where N was less than the length of the first
      available chunk would mutate the deque instance twice and allocate a new
      str from the slice of the existing chunk. Profiling drawed my attention
      to these as a potential hot spot during changegroup reading.
      
      This patch makes the code more complicated in order to avoid the
      aforementioned 3 operations.
      
      On a pre-generated mozilla-central gzip bundle, this series has the
      following impact on `hg unbundle` performance on my MacBook Pro:
      
      before: 358.21 real       317.69 user        38.49 sys
      after:  301.57 real       262.69 user        37.11 sys
      delta:  -56.64 real       -55.00 user        -1.38 sys
      6ae14d1c
  12. Oct 05, 2015
  13. Sep 25, 2015
    • Siddharth Agarwal's avatar
      localrepo: allow wlock to be inherited · efd57cd6
      Siddharth Agarwal authored
      This is part of a series that will allow locks to be inherited by subprocesses
      in limited circumstances.
      
      When allowed, the parent process will pass down requisite information to the
      child process by way of this environment variable.
      efd57cd6
  14. Oct 05, 2015
  15. Oct 04, 2015
    • Gregory Szorc's avatar
      exchange: add "streaming all changes" to bundle2 pulling · 41dd7b2c
      Gregory Szorc authored
      This is the beginning of client-side support for performing a stream
      clone using bundle2. The main bundle2 pull function checks whether to
      perform a streaming clone and outputs a message if so.
      
      While we have a duplicate message, it seems easier to have all the
      bundle2 console writing in one location and in an easy-to-read
      conditional block.
      41dd7b2c
    • Gregory Szorc's avatar
      streamclone: move "streaming all changes" message location · 4b5647d9
      Gregory Szorc authored
      Previously, the message was printed after we requested and started
      processing the remote stream. This seems like something that we should
      do before calling out to the remote. Moving it also makes it easier to
      deal with the bundle2 implementation.
      4b5647d9
  16. Oct 05, 2015
    • Gregory Szorc's avatar
      streamclone: move payload header generation into own function · fb743268
      Gregory Szorc authored
      The stream clone data over the wire protocol contains a header line
      indicating total file count and data size. In bundle2, this metadata can
      be captured by a part parameter and doesn't need to be in the body.
      
      In preparation for bundle2, have generatev1() return the raw metadata
      and move the header generation to its own function.
      fb743268
    • Gregory Szorc's avatar
      streamclone: move payload header line consumption · 19bbd53a
      Gregory Szorc authored
      bundle2 parts have parameters. These are a logical place for "header"
      data such as the file count and payload size of stream clone data. In
      preparation for supporting stream clones with bundle2, move the
      consumption of the header line from the payload into
      maybeperformlegacystreamclone().
      
      Note: the header line is still being emitted by generatev1(). This will
      be addressed in a subsequent patch.
      19bbd53a
    • Gregory Szorc's avatar
      streamclone: teach canperformstreamclone to be bundle2 aware · ff2c8923
      Gregory Szorc authored
      We add an argument to canperformstreamclone() to return False if a
      bundle2 stream clone is available. This will enable the legacy stream
      clone step to no-op when a bundle2 stream clone is supported.
      
      The commented code will be made active when bundle2 supports streaming
      clone.
      
      This patch does foreshadow the introduction of the "stream" bundle2
      capability and its "v1" sub-capability. The bundle2 capability mirrors
      the existing "stream" capability and is needed so clients know whether a
      server explicitly supports streaming clones over bundle2 (servers up to
      this point support bundle2 without streaming clone support).
      
      The sub-capability will denote which data formats and variations are
      supported. Currently, the value "v1" denotes the existing streaming
      clone data format, which I intend to reuse inside a bundle2 part. My
      intent is to eventually introduce alternate data formats that can be
      produced and consumed more efficiently. Having a sub-capability means
      we don't need to introduce a new top-level bundle2 capability when new
      formats are introduced. This doesn't really have any implications
      beyond making the capabilities namespace more organized.
      ff2c8923
  17. Oct 04, 2015
  18. Oct 05, 2015
    • Gregory Szorc's avatar
      exchange: expose bundle2 capabilities on pulloperation · 9a7fc6d4
      Gregory Szorc authored
      This adds a cache and makes accessing the capabilities slightly simpler,
      as you don't need to directly go through the bundle2 module. This will
      also help prevent a function-level import in streamclone.py.
      
      This patch arguably isn't necessary. But I think it makes things
      slightly nicer.
      9a7fc6d4
  19. Oct 04, 2015
    • Katsunori FUJIWARA's avatar
      keyword: make restrict mode False while updating files for rollback · b3188339
      Katsunori FUJIWARA authored
      This is a preparation for using 'repo.rollback()' instead of aborting
      a current running transaction for "shelve" and "unshelve".
      
      Before this patch, updating files as a part of 'repo.rollback()'
      overridden by keyword extension always follows 'restrict' mode of the
      command currently executed.
      
      "merge", "unshelve" and so on should be 'restrict'-ed, because keyword
      expansion may cause unexpected conflicts at merging while these
      commands.
      
      But, if 'repo.rollback()' is invoked while executing 'restrict'-ed
      commands, modified files in the working directory are marked as
      "CLEAN" unexpectedly by code path below:
      
              # 'lookup' below is True at updating modified files for rollback
              kwcmd = self.restrict and lookup # kwexpand/kwshrink
                  :
                      if kwcmd:
                          self.repo.dirstate.normal(f)
      
      On the other hand, "rollback" command isn't 'restrict'-ed, because
      rollbacking itself doesn't imply merging.
      
      Therefore, disabling 'restrict' mode while updating files as a part of
      'repo.rollback()' regardless of current 'restrict' mode should be
      reasonable.
      b3188339
    • Gregory Szorc's avatar
      streamclone: rename and document maybeperformstreamclone() · 3b0ec091
      Gregory Szorc authored
      Upcoming patches will introduce bundle2 based streaming clones. Add
      "legacy" to the function name and add a docstring clarifying the intent of
      the function.
      3b0ec091
    • Gregory Szorc's avatar
      streamclone: move applyremotedata() into maybeperformstreamclone() · 09cc3c2e
      Gregory Szorc authored
      Future work around stream cloning will be implemented in a bundle2
      world. This code will only be used in the legacy code path and
      doesn't need to be abstracted or extensible.
      09cc3c2e
  20. Oct 03, 2015
    • Gregory Szorc's avatar
      branchmap: move branch cache code out of streamclone.py · 79ef8675
      Gregory Szorc authored
      This is low-level branch map and cache manipulation code. It deserves to
      live next to similar code in branchmap.py. Moving it also paves the road
      for multiple consumers, such as a bundle2 part handler that receives
      branch mappings from a remote.
      
      This is largely a mechanical move, with only variable names and
      indentation being changed.
      79ef8675
    • Gregory Szorc's avatar
      streamclone: move streamin() into maybeperformstreamclone() · 3b28ffde
      Gregory Szorc authored
      streamin() only had a single consumer. And it always only ever will
      because it is strongly coupled with the current,
      soon-to-be-superseded-by-bundle2 functionality.
      
      The return value has been dropped because nobody was using it.
      3b28ffde
  21. Oct 04, 2015
    • Gregory Szorc's avatar
      streamclone: refactor maybeperformstreamclone to take a pullop · 36279329
      Gregory Szorc authored
      Just like all the other pull steps. Consistency is good.
      
      This seems a little excessive right now since maybeperformstreamclone is
      such a short function. This will be addressed in a subsequent patch.
      36279329
    • Gregory Szorc's avatar
      demandimport: replace more references to _demandmod instances · 7e813050
      Gregory Szorc authored
      _demandmod instances may be referenced by multiple importing modules.
      Before this patch, the _demandmod instance only maintained a reference
      to its first consumer when using the "from X import Y" syntax. This is
      because we only created a single _demandmod instance (attached to the
      parent X module). If multiple modules A and B performed
      "from X import Y", we'd produce a single _demandmod instance
      "demandmod" with the following references:
      
        X.Y = <demandmod>
        A.Y = <demandmod>
        B.Y = <demandmod>
      
      The locals from the first consumer (A) would be stored in <demandmod1>.
      When <demandmod1> was loaded, we'd look at the locals for the first
      consumer and replace the symbol, if necessary. This resulted in state:
      
        X.Y = <module>
        A.Y = <module>
        B.Y = <demandmod>
      
      B's reference to Y wasn't updated and was still using the proxy object
      because we just didn't record that B had a reference to <demandmod> that
      needed updating!
      
      With this patch, we add support for tracking which modules in addition
      to the initial importer have a reference to the _demandmod instance and
      we replace those references at module load time.
      
      In the case of posix.py, this fixes an issue where the "encoding" module
      was being proxied, resulting in hundreds of thousands of
      __getattribute__ lookups on the _demandmod instance during dirstate
      operations on mozilla-central, speeding up execution by many
      milliseconds. There are likely several other operation that benefit from
      this change as well.
      
      The new mechanism isn't perfect: references in locals (not globals) may
      likely linger. So, if there is an import inside a function and a symbol
      from that module is used in a hot loop, we could have unwanted overhead
      from proxying through _demandmod. Non-global imports are discouraged
      anyway. So hopefully this isn't a big deal in practice. We could
      potentially deploy a code checker that bans use of attribute lookups of
      function-level-imported modules inside loops.
      
      This deficiency in theory could be avoided by storing the set of globals
      and locals dicts to update in the _demandmod instance. However, I tried
      this and it didn't work. One reason is that some globals are _demandmod
      instances. We could work around this, but it's a bit more work. There
      also might be other module import foo at play. The solution as
      implemented is better than what we had and IMO is good enough for the
      time being.
      
      It's worth noting that this sub-optimal behavior was made worse by the
      introduction of absolute_import and its recommended "from . import X"
      syntax for importing modules from the "mercurial" package. If we ever
      wrote performance tests, measuring the amount of module imports and
      __getattribute__ proxy calls through _demandmod instances would be
      something I'd have it check.
      7e813050
    • Gregory Szorc's avatar
      demandimport: refactor processfromitem · 86fc4a28
      Gregory Szorc authored
      This will match the next patch smaller and easier to read.
      86fc4a28
Loading