Skip to content
Snippets Groups Projects
  1. Apr 02, 2018
  2. Mar 17, 2018
    • Yuya Nishihara's avatar
      templater: pass (context, mapping) down to unwraphybrid() · 7d3bc1d4
      Yuya Nishihara authored
      See the subsequent patches for why.
      
      I initially thought it would be wrong to pass a mapping to flatten() and
      stringify() since these functions may be applied to a tree of generators,
      where each node should be bound to the mapping when it was evaluated. But,
      actually that isn't a problem. If an intermediate node has to override a
      mapping dict, it can do on unwraphybrid() and yield "unwrapped" generator
      of byte strings:
      
         "{f(g(v))}"  # literal template example.
             ^^^^     # g() want to override a mapping, so it returns a wrapped
                      # object 'G{V}' with partial mapping 'lm' attached.
           ^^^^^^^    # f() stringifies 'G{V}', starting from a mapping 'm'.
                      # when unwrapping 'G{}', it updates 'm' with 'lm', and
                      # passes it to 'V'.
      
      This structure is important for the formatter (and the hgweb) to build a
      static template keyword, which can't access a mapping dict until evaluation
      phase.
      7d3bc1d4
  3. Apr 02, 2018
  4. Apr 01, 2018
  5. Mar 31, 2018
    • Matt Harbison's avatar
      lfs: add an experimental knob to disable blob serving · dfb38c48
      Matt Harbison authored
      The use case here is the server admin may want to store the blobs elsewhere.  As
      it stands now, the `lfs.url` config on the client side is all that enforces this
      (the web.allow-* permissions aren't able to block LFS blobs without also
      blocking normal hg traffic).  The real solution to this is to implement the
      'verify' action on the client and server, but that's not a near term goal.
      Whether this is useful in its own right, and should be promoted out of
      experimental at some point is TBD.
      
      Since the other two tests that deal with LFS and `hg serve` are already complex
      and have #testcases, this seems like a good time to start a new test dedicated
      to access checks against the server.  Instead of conditionally wrapping the
      wire protocol handler, I put this in the handler because I'd still like to bring
      the annotations in from the evolve extension in order to set up the wrapping.
      The 400 status probably isn't great, but that's what it would be for existing
      `hg serve` instances without support for serving blobs.
      dfb38c48
  6. Feb 26, 2018
    • Matt Harbison's avatar
      lfs: improve the client message when the server signals an object error · 67db8484
      Matt Harbison authored
      Two things here.  First, the previous message included a snippet of JSON, which
      tends to be long (and in the case of lfs-test-server, has no error message).
      Instead, give a concise message where possible, and leave the JSON to a debug
      output.  Second, the server can signal issues other than a missing individual
      file.  This change shows a corrupt file, but I'm debating letting the corrupt
      file get downloaded, because 1) the error code doesn't really fit, and 2) having
      it locally makes forensics easier.  Maybe need a config knob for that.
      67db8484
  7. Mar 31, 2018
    • Pulkit Goyal's avatar
      infinitepush: add tests for `hg pull -r <rev>` · b5caa13d
      Pulkit Goyal authored
      This patch adds test for `hg pull -r <rev>` when the infinitepush extension is
      not present on the client side and the server by defaults pushes all the
      incoming push to bundlestore.
      
      As the tests show, if a the changeset was applied to the server, that can be
      pulled, but if a changeset is in the bundlestore we cannot pull that yet. We
      should support that.
      
      Differential Revision: https://phab.mercurial-scm.org/D2998
      b5caa13d
  8. Mar 18, 2018
  9. Mar 25, 2018
  10. Mar 31, 2018
  11. Mar 19, 2018
    • Connor Sheehan's avatar
      templatefuncs: add mailmap template function · 2a2ce93e
      Connor Sheehan authored
      This commit adds a template function to support the .mailmap file
      in Mercurial repositories. The .mailmap file comes from git, and
      can be used to map new emails and names for old commits. The general
      use case is that someone may change their name or author commits
      under different emails and aliases, which would make these
      commits appear as though they came from different persons. The
      file allows you to specify the correct name that should be used
      in place of the author field specified in the commit.
      
      The mailmap file has 4 possible formats used to map old "commit"
      names to new "proper" names:
      
      1. <proper@email.com> <commit@email.com>
      2. Proper Name <commit@email.com>
      3. Proper Name <proper@email.com> <commit@email.com>
      4. Proper Name <proper@email.com> Commit Name <commit@email.com>
      
      Essentially there is a commit email present in each mailmap entry,
      that maps to either an updated name, email, or both. The final
      possible format allows commits authored by a person who used
      both an old name and an old email to map to a new name and email.
      
      To parse the file, we split by spaces and build a name out
      of every element that does not start with "<". Once we find an element
      that does start with "<" we concatenate all the name elements that preceded
      and add that as a parsed name. We then add the email as the first
      parsed email. We repeat the process until the end of the line, or
      a comment is found. We will be left with all parsed names in a list,
      and all parsed emails in a list, with the 0 index being the proper
      values and the 1 index being the commit values (if they were specified
      in the entry).
      
      The commit values are added as the keys to a dict, and with the proper
      fields as the values. The mapname function takes the mapping object and
      the commit author field and attempts to look for a corresponding entry.
      To do so we try (commit name, commit email) first, and if no results are
      returned then (None, commit email) is also looked up. This is due to
      format 4 from above, where someone may have a mailmap entry with both
      name and email, and if they don't it is possible they have an entry that
      uses only the commit email.
      
      Differential Revision: https://phab.mercurial-scm.org/D2904
      2a2ce93e
  12. Mar 30, 2018
  13. Mar 27, 2018
    • Pulkit Goyal's avatar
      infinitepush: introduce server option to route every push to bundlestore · 571f25da
      Pulkit Goyal authored
      This patch introduces a new config option for server
      `infinitepush.pushtobundlestore` which if sets to True, the server will route
      each incoming push to the bundlestore and store all the parts i.e. changegroups,
      phases, obsmarkers in the bundlestore and won't be applied to the revlog.
      
      This config option does not need any client side wrapping and does not need any
      custom bundle2 part or stream level parameter to decide where the push should
      go.
      
      This is very useful for Mozilla CI use case where they have a central server
      that recieves pushes to trigger code-reviews, trigger a test run of CI, run
      static analysis etc. The server using the new config option can stash standalone
      bundles to the bundlestore and server can get access to individual revisions on
      demand.
      
      A new test file which has related tests are added for the config option.
      
      Differential Revision: https://phab.mercurial-scm.org/D2958
      571f25da
  14. Mar 30, 2018
  15. Mar 21, 2018
  16. Mar 30, 2018
  17. Mar 27, 2018
  18. Feb 09, 2018
    • Pulkit Goyal's avatar
      infinitepush: drop the `--to` flag to push and use `-B` instead · c5687ce3
      Pulkit Goyal authored
      The extension added a `--to` flag to specify the bookmark to which revs should
      be pushed. This patch deletes that flag and instead uses the `-B` flag. After
      this patch, bookmark passed as `-B` is parsed and if it matches the infinitepush
      bookmark pattern, we consider that push as infinitepush.
      
      This is still not the best of what we can do. Later patches in the series will
      drop the use of `-B` flag and will instead handle things at bookmark bundle2
      part. Plugging these logic to bookmark bundle2 part will also get rid of the
      scratchbranchparttype bundle2 part.
      
      Differential Revision: https://phab.mercurial-scm.org/D2108
      c5687ce3
    • Pulkit Goyal's avatar
      infinitepush: drop the `--create` flag to push command · 78c2e941
      Pulkit Goyal authored
      The extension added a `--create` flag for creating a new bookmark on the
      bundlestore. This patch changes the bahviour to create a bookmark if it does not
      exists and removes the requirement of the `--create` and then drop the logic
      around the `--create` flag.
      
      Tests are changed to drop the usage of `--create` flag.
      
      Differential Revision: https://phab.mercurial-scm.org/D2107
      78c2e941
    • Pulkit Goyal's avatar
      infinitepush: drop error handling while pushing to svn server · 3be6f435
      Pulkit Goyal authored
      This is something which is internal to Facebook and we don't want in core.
      
      Differential Revision: https://phab.mercurial-scm.org/D2104
      3be6f435
    • Pulkit Goyal's avatar
      infinitepush: drop `--list-remote`, `--remote-path` flags to bookmark cmd · 320b1f95
      Pulkit Goyal authored
      The extension adds these two flags to list remote bookmarks using the `hg
      bookmark` command. These are not required in core currently and needs some
      discussion before getting them in, so let's drop them for now. This makes us
      deleting the wrapping of `hg bookmark` command on client side.
      
      The end goal here is to have minimal or no wrapping at client side.
      
      Differential Revision: https://phab.mercurial-scm.org/D2103
      320b1f95
    • Pulkit Goyal's avatar
      infinitepush: move the extension to core from fb-hgext · 03ff17a4
      Pulkit Goyal authored
      This patch moves the infinitepush extension from fb-hgext to core. The
      extension is used to store incoming bundles during a push in bundlestore rather
      than applying them to the revlog.
      
      The extension was copied from the repository revision at
      f27f094e91553d3cae5167c0b1c42ae940f888d5 and following changes were made:
      
        * added `from __future__ import absolute_import` where missing
        * fixed module imports to follow the core style
        * minor fixes for test-check-code.t
        * registered the configs
        * adding the testedwith value to match core's convention
        * removed double newlines to make test-check-commit.t happy
        * added one line doc about extension and marked it as experimental
      
      Only one test file test-infinitepush-bundlestore.t is moved to core and
      following changes are made to file:
      
        * remove dependency of library.sh
        * split the tests into two tests i.e. test-infinitepush.t and
          test-infinitepush-bundlestore.t
        * removed testing related to other facebook's extensions pushrebase, inhibit,
          fbamend
      
      library-infinitepush.sh is also copied from fb-hgext from the same revision and
      following changes are made:
      
        * change the path to infinitepush extension as it's in core with this patch
        * removed sql handling from the file as we are not testing that initially
      
      Currently at this revision, test-check-module-imports.t does not pass as there
      is import of a module from fb/hgext in one the of the file which will be removed
      in the next patch.
      
      This extension right now has a lot of things which we don't require in core like
      `--to`, `--create` flags to `hg bookmark`, logic related to remotenames
      extension and another facebook's extensions, custom bundle2parts which can be
      prevented by using bookmarks bundle part and also logic related to sql store
      which is probably we don't want initially.
      
      The next patches in this series will remove all the unwanted and unrequired
      things from the extension and will make this a nice one.
      
      The end goal is to have a very lighweight extension with no or very less
      wrapping on the client side.
      
      Differential Revision: https://phab.mercurial-scm.org/D2096
      03ff17a4
  19. Mar 30, 2018
  20. Mar 03, 2018
  21. Mar 30, 2018
  22. Mar 22, 2018
    • Gregory Szorc's avatar
      repository: define interface for local repositories · 0dfb5672
      Gregory Szorc authored
      Per discussions on the mailing list and at the 4.4 and 4.6 sprints,
      we want to start defining interfaces for local repository primitives
      so that we a) have a better idea of what the formal interface for
      various types is b) can more easily introduce alternate implementations
      of various components (e.g. in Rust).
      
      We have previously implemented interfaces that declare the peer and
      wire protocol APIs using the abc module.
      
      This commit introduces a monolithic interface for the localrepository
      class. It uses zope.interface - not abc - for defining and declaring
      the interface.
      
      The newly defined "completelocalrepository" interface is objectively
      horrible. It is based on what is actually in localrepository and
      doesn't represent a reasonable interface definition IMO. There's lots
      of... unwanted garbage in the interface. In other words, it reflects
      the horrible state of the localrepository "god object." But this is
      fine: a goal of this commit is to get the interface defined so that
      we have an interface. Future commits can refactor the interface
      into sub-interfaces, remove unwanted public attributes, etc.
      
      I attempted to define reasonable docstrings for the various interface
      members. But there are so many of them and I didn't know what some are
      used for. So I was lazy in a number of places and didn't write
      docstrings or detailed usage docs.
      
      Also, the members of the interface are defined in the order they are
      declared in localrepo.py. This revealed that the grouping of things
      in localrepo.py is... odd.
      
      The localrepository class now declares that it implements our newly
      defined interface. Unlike abc, zope.interface doesn't check interface
      conformance at type creation time (abc uses __metaclass__ magic to
      validate interface conformance when a type is created - usually at
      module import time). It does provide some functions for validating
      class and object conformance with declared interfaces. We add these
      checks to test-check-interfaces.py. We /could/ validate at run-time.
      But we hold off for now. (I'm a bit scared of doing that because of
      the various ways extensions monkeypatch repo instances.)
      
      After this commit, test-check-interfaces.py will fail if the set of
      public attributes on the localrepository class or instances change
      without corresponding updates to the interface. This is by design.
      
      Differential Revision: https://phab.mercurial-scm.org/D2933
      0dfb5672
  23. Jan 31, 2018
  24. Mar 30, 2018
  25. Mar 17, 2018
    • Matt Harbison's avatar
      test-lfs-test-server: add a testcase for `hg serve` · f51c2780
      Matt Harbison authored
      I haven't figured out yet how to make the authentication checks work for a
      specific list of users, so the 'web.allow-push' list is wildcarded.  (It appears
      that the client doesn't react to a 401 by sending authentication data, which may
      be caused in part by not having all of the headers in httpbasicauthhandler's
      http_error_auth_reqed(), compared to a run of test-http.t.  But in any case, we
      should probably have a separate set of tests for various authentication
      scenarios.  As it is, without the wildcard, no push access is granted.)
      
      There are several deviations from the `lfs-test-server` case:
      
       - `hg serve` emits a Server header.  I think Gregory indicated that this isn't
         easily suppressed.
      
       - `hg serve` names the "basic" transfer handler in the Batch API response.  Not
         having to specify it was for backwards compatability, so this seems like the
         right thing to do.  (`lfs-test-server` doesn't name it, whether it was
         explicitly requested by the client or not.)
      
       - PUT status for a newly created file is 201, per RFC-2616 [1].  The Basic
         Transfer API [2] shows an example upload transcript with a 200 response.  It
         doesn't make much sense to re-upload a file (unless it is corrupt) in an
         example, but I wouldn't be surprised if some other implementations also
         expect 200 because of this.  But the RFC says MUST use 201 for creation.
      
       - The Content-Type for the file transfers is "application/octet-stream", like
         the sample transcript (though I don't see it explicitly called out in the
         text elsewhere).  Using "text/plain" seems clearly wrong.
      
       - `lfs-test-server` isn't removing the action property and sending back an
         error code like the spec calls out when a file is missing or corrupt.  Doing
         so on the `hg serve` side reveals a bug in our client code when handling the
         response- it indicates the remote file is missing instead of corrupt around
         line 452.
      
      I'll probably glob over the Content-Length differences once this settles down.
      Prior to the recent hgweb refactoring, the Batch API response was using chunked
      encodings instead.
      
      Back to the RFC, I have no idea if the python framework handles the "MUST NOT
      ignore any Content-* (e.g. Content-Range) headers that it does not understand or
      implement and MUST return a 501" for a PUT request.
      
      [1] https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6
      [2] https://github.com/git-lfs/git-lfs/blob/master/docs/api/basic-transfers.md#uploads
      f51c2780
  26. Mar 21, 2018
  27. Mar 15, 2018
    • Matt Harbison's avatar
      test-lfs: drop trailing ', ' item separators from debug JSON output · c37c47e4
      Matt Harbison authored
      The trailing space looks weird when conditionalizing the line.  The commas
      shouldn't be necessary because of the indenting.  The `lfs-test-server` isn't
      sending all of the same items (notably, the "transfer" attribute is missing), so
      having the commas means more lines need to be conditionalized.
      c37c47e4
Loading