Skip to content
Snippets Groups Projects
  1. Mar 17, 2018
    • Matt Harbison's avatar
      test-lfs-test-server: add a testcase for `hg serve` · f51c2780db3a
      Matt Harbison authored
      I haven't figured out yet how to make the authentication checks work for a
      specific list of users, so the 'web.allow-push' list is wildcarded.  (It appears
      that the client doesn't react to a 401 by sending authentication data, which may
      be caused in part by not having all of the headers in httpbasicauthhandler's
      http_error_auth_reqed(), compared to a run of test-http.t.  But in any case, we
      should probably have a separate set of tests for various authentication
      scenarios.  As it is, without the wildcard, no push access is granted.)
      
      There are several deviations from the `lfs-test-server` case:
      
       - `hg serve` emits a Server header.  I think Gregory indicated that this isn't
         easily suppressed.
      
       - `hg serve` names the "basic" transfer handler in the Batch API response.  Not
         having to specify it was for backwards compatability, so this seems like the
         right thing to do.  (`lfs-test-server` doesn't name it, whether it was
         explicitly requested by the client or not.)
      
       - PUT status for a newly created file is 201, per RFC-2616 [1].  The Basic
         Transfer API [2] shows an example upload transcript with a 200 response.  It
         doesn't make much sense to re-upload a file (unless it is corrupt) in an
         example, but I wouldn't be surprised if some other implementations also
         expect 200 because of this.  But the RFC says MUST use 201 for creation.
      
       - The Content-Type for the file transfers is "application/octet-stream", like
         the sample transcript (though I don't see it explicitly called out in the
         text elsewhere).  Using "text/plain" seems clearly wrong.
      
       - `lfs-test-server` isn't removing the action property and sending back an
         error code like the spec calls out when a file is missing or corrupt.  Doing
         so on the `hg serve` side reveals a bug in our client code when handling the
         response- it indicates the remote file is missing instead of corrupt around
         line 452.
      
      I'll probably glob over the Content-Length differences once this settles down.
      Prior to the recent hgweb refactoring, the Batch API response was using chunked
      encodings instead.
      
      Back to the RFC, I have no idea if the python framework handles the "MUST NOT
      ignore any Content-* (e.g. Content-Range) headers that it does not understand or
      implement and MUST return a 501" for a PUT request.
      
      [1] https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6
      [2] https://github.com/git-lfs/git-lfs/blob/master/docs/api/basic-transfers.md#uploads
      f51c2780db3a
  2. Mar 29, 2018
  3. Mar 21, 2018
  4. Mar 17, 2018
    • Matt Harbison's avatar
      lfs: add support for serving blob files · cc0a6ea95d98
      Matt Harbison authored
      cc0a6ea95d98
    • Matt Harbison's avatar
      lfs: add server side support for the Batch API · ea6fc58524d7
      Matt Harbison authored
      ea6fc58524d7
    • Matt Harbison's avatar
      lfs: add basic routing for the server side wire protocol processing · a2566597acb5
      Matt Harbison authored
      The recent hgweb refactoring yielded a clean point to wrap a function that could
      handle this, so I moved the routing for this out of the core.  While not an hg
      wire protocol, this seems logically close enough.  For now, these handlers do
      nothing other than check permissions.
      
      The protocol requires support for PUT requests, so that has been added to the
      core, and funnels into the same handler as GET and POST.  The permission
      checking code was assuming that anything not checking 'pull' or None ops should
      be using POST.  But that breaks the upload check if it checks 'push'.  So I
      invented a new 'upload' permission, and used it to avoid the mandate to POST.  A
      function wrap point could be added, but security code should probably stay
      grouped together.  Given that anything not 'pull' or None was requiring POST,
      the comment on hgweb.common.permhooks is probably wrong- there is no 'read'.
      
      The rationale for the URIs is that the spec for the Batch API[1] defines the URL
      as the LFS server url + '/objects/batch'.  The default git URLs are:
      
          Git remote: https://git-server.com/foo/bar
          LFS server: https://git-server.com/foo/bar.git/info/lfs
          Batch API: https://git-server.com/foo/bar.git/info/lfs/objects/batch
      
      '.git/' seems like it's not something a user would normally track.  If we adhere
      to how git defines the URLs, then the hg-git extension should be able to talk to
      a git based server without any additional work.
      
      The URI for the transfer requests starts with '.hg/' to ensure that there are no
      conflicts with tracked files.  Since these are handed out by the Batch API, we
      can change this at any point in the future.  (Specifically, it might be a good
      idea to use something under the proposed /api/ namespace.)  In any case, no
      files are stored at these locations in the repository directory.
      
      I started a new module for this because it seems like a good idea to keep all of
      the security sensitive server side code together.  There's also an issue with
      `hg verify` in that it will want to download *all* blobs in order to run.
      Sadly, there's no way in the protocol to ask the server to verify the content of
      a blob it may have.  (The verify action is for storing files on a 3rd party
      server, and then informing the LFS server when that completes.)  So we may end
      up implementing a custom transfer adapter that simply indicates if the blobs are
      valid, and fall back to basic transfers for non-hg servers.  In other words,
      this code is likely to get bigger before this is made non-experimental.
      
      [1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md
      a2566597acb5
  5. Mar 15, 2018
    • Matt Harbison's avatar
      test-lfs: drop trailing ', ' item separators from debug JSON output · c37c47e47a95
      Matt Harbison authored
      The trailing space looks weird when conditionalizing the line.  The commas
      shouldn't be necessary because of the indenting.  The `lfs-test-server` isn't
      sending all of the same items (notably, the "transfer" attribute is missing), so
      having the commas means more lines need to be conditionalized.
      c37c47e47a95
  6. Feb 23, 2018
  7. Mar 29, 2018
  8. Mar 28, 2018
  9. Mar 27, 2018
    • Matt Harbison's avatar
      lfs: respect narrowmatcher when testing to add 'lfs' requirement (issue5794) · 4d63f3bc1e1a
      Matt Harbison authored
      There's a similar test in lfs.wrapper.convertsink(), but I didn't update that
      because I don't think that the sink repo in a convert can be narrow.
      
      It seems reasonable that a narrow clone of an LFS repo may not necessarily be an
      LFS repo.  The only potential issue is that LFS has a hard requirement for
      changegroup v3, which that extension enables.  The use of treemanifest will
      enable changegroup v3 in narrow clones, because allsupportedversions() in
      changegroup.py preserves it when it sees a 'treemanifest' requirement.  But I
      don't see where changegroup v3 is enabled for a flat manifest.
      4d63f3bc1e1a
  10. Mar 21, 2018
  11. Mar 04, 2018
  12. Mar 27, 2018
    • Sangeet Kumar Mishra's avatar
      grep: fixes erroneous output of grep in forward order (issue3885) · a2a6755a3def
      Sangeet Kumar Mishra authored
      If grep is passed a revset in forwards order via -r , say -r 0:tip
      Then the output is erroneous. This patch fixes that. The output was wrong
      because we deleted the last revision key in the matches and when we moved
      to the next revision we didn't had this to compare the diff. So the pstates
      dict was always empty and in the SequenceMatcher, to convert and empty pstate
      to the states dictionary you would always insert. This patch keeps the matches
      dictionary until the end of this window and clears it at once when this
      window ends. This solves the above mentioned problem and also do not cause
      any memory leak.
      a2a6755a3def
    • Matt Harbison's avatar
      lfs: move the 'supportedoutgoingversions' handling to changegroup.py · a54113fcc8c9
      Matt Harbison authored
      This handling already exists here for the narrow extension.  We still need to
      either figure out how to enable changegroup v3 without the extension, or figure
      out how to let the server detect that the client doesn't have it loaded, and
      emit a user friendly error[1].  I can't tell if D1944 is the appropriate vehicle
      for the latter.
      
      [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-January/109550.html
      a54113fcc8c9
  13. Mar 04, 2018
  14. Mar 26, 2018
  15. Mar 05, 2018
  16. Mar 25, 2018
  17. Mar 24, 2018
  18. Mar 25, 2018
Loading