Skip to content
Snippets Groups Projects
Commit 3790efb388ca authored by Matt Harbison's avatar Matt Harbison
Browse files

lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)

There are only a handful of methods that access repo attributes that are applied
in reposetup().  The `diff` test covers all of the commands that call
scmutil.prefetchfiles().  Along the way, I saw that adding files and upgrading
the repo format were also problems (also tested here).

I don't think running `hg serve` through the commandserver is sane, but I
conditionalized both the capabilities and the wsgirequest handler because it's
trivially correct.  It doesn't look like there has ever been a caller of
candownload(), so there's no test for that path.

The upload case isn't testable, because uploadblobs() bails if there are no
pointers.  The requirement should be added any time pointers are introduced, and
that would force the extension to be loaded specifically for the repo.  This
covers `debuglfsupload`, the pre-push hook (which isn't set until the repo is
promoted to LFS), and uploadblobsfromrevs(), which can be called by other
extensions.

I think readfromstore() and writetostore() are only reachable as a flag
processor for revlog.REVIDX_EXTSTORED, and a requirement is added as soon as
that is seen, so I don't think those are a problem.
parent a3b4ccbec269
No related branches found
No related tags found
No related merge requests found
......@@ -18,6 +18,7 @@
from mercurial import (
pycompat,
util,
)
from . import blobstore
......@@ -40,6 +41,9 @@
if not rctx.repo.ui.configbool('experimental', 'lfs.serve'):
return False
if not util.safehasattr(rctx.repo.svfs, 'lfslocalblobstore'):
return False
if not req.dispatchpath:
return False
......
......@@ -37,8 +37,9 @@
def _capabilities(orig, repo, proto):
'''Wrap server command to announce lfs server capability'''
caps = orig(repo, proto)
# XXX: change to 'lfs=serve' when separate git server isn't required?
caps.append('lfs')
if util.safehasattr(repo.svfs, 'lfslocalblobstore'):
# XXX: change to 'lfs=serve' when separate git server isn't required?
caps.append('lfs')
return caps
def bypasscheckhash(self, text):
......@@ -118,9 +119,6 @@
def filelogaddrevision(orig, self, text, transaction, link, p1, p2,
cachedelta=None, node=None,
flags=revlog.REVIDX_DEFAULT_FLAGS, **kwds):
textlen = len(text)
# exclude hg rename meta from file size
meta, offset = revlog.parsemeta(text)
if offset:
textlen -= offset
# The matcher isn't available if reposetup() wasn't called.
lfstrack = self.opener.options.get('lfstrack')
......@@ -126,3 +124,8 @@
lfstrack = self.opener.options['lfstrack']
if lfstrack:
textlen = len(text)
# exclude hg rename meta from file size
meta, offset = revlog.parsemeta(text)
if offset:
textlen -= offset
......@@ -128,6 +131,6 @@
if lfstrack(self.filename, textlen):
flags |= revlog.REVIDX_EXTSTORED
if lfstrack(self.filename, textlen):
flags |= revlog.REVIDX_EXTSTORED
return orig(self, text, transaction, link, p1, p2, cachedelta=cachedelta,
node=node, flags=flags, **kwds)
......@@ -247,6 +250,9 @@
def _prefetchfiles(repo, revs, match):
"""Ensure that required LFS blobs are present, fetching them as a group if
needed."""
if not util.safehasattr(repo.svfs, 'lfslocalblobstore'):
return
pointers = []
oids = set()
localstore = repo.svfs.lfslocalblobstore
......@@ -266,7 +272,11 @@
blobstore.remote(repo).readbatch(pointers, localstore)
def _canskipupload(repo):
# Skip if this hasn't been passed to reposetup()
if not util.safehasattr(repo.svfs, 'lfsremoteblobstore'):
return True
# if remotestore is a null store, upload is a no-op and can be skipped
return isinstance(repo.svfs.lfsremoteblobstore, blobstore._nullremote)
def candownload(repo):
......@@ -269,7 +279,11 @@
# if remotestore is a null store, upload is a no-op and can be skipped
return isinstance(repo.svfs.lfsremoteblobstore, blobstore._nullremote)
def candownload(repo):
# Skip if this hasn't been passed to reposetup()
if not util.safehasattr(repo.svfs, 'lfsremoteblobstore'):
return False
# if remotestore is a null store, downloads will lead to nothing
return not isinstance(repo.svfs.lfsremoteblobstore, blobstore._nullremote)
......@@ -389,6 +403,9 @@
def upgradefinishdatamigration(orig, ui, srcrepo, dstrepo, requirements):
orig(ui, srcrepo, dstrepo, requirements)
srclfsvfs = srcrepo.svfs.lfslocalblobstore.vfs
dstlfsvfs = dstrepo.svfs.lfslocalblobstore.vfs
# Skip if this hasn't been passed to reposetup()
if (util.safehasattr(srcrepo.svfs, 'lfslocalblobstore') and
util.safehasattr(dstrepo.svfs, 'lfslocalblobstore')):
srclfsvfs = srcrepo.svfs.lfslocalblobstore.vfs
dstlfsvfs = dstrepo.svfs.lfslocalblobstore.vfs
......@@ -394,8 +411,8 @@
for dirpath, dirs, files in srclfsvfs.walk():
for oid in files:
ui.write(_('copying lfs blob %s\n') % oid)
lfutil.link(srclfsvfs.join(oid), dstlfsvfs.join(oid))
for dirpath, dirs, files in srclfsvfs.walk():
for oid in files:
ui.write(_('copying lfs blob %s\n') % oid)
lfutil.link(srclfsvfs.join(oid), dstlfsvfs.join(oid))
def upgraderequirements(orig, repo):
reqs = orig(repo)
......
......@@ -23,6 +23,15 @@
. | E || #3 | #3 | X | #6 |
. |---++-----------------------+
make command server magic visible
#if windows
$ PYTHONPATH="$TESTDIR/../contrib;$PYTHONPATH"
#else
$ PYTHONPATH="$TESTDIR/../contrib:$PYTHONPATH"
#endif
$ export PYTHONPATH
$ hg init server
$ SERVER_REQUIRES="$TESTTMP/server/.hg/requires"
......@@ -55,9 +64,19 @@
$ grep 'lfs' client/.hg/requires $SERVER_REQUIRES
[1]
This trivial repo will force commandserver to load the extension, but not call
reposetup() on another repo actually being operated on. This gives coverage
that wrapper functions are not assuming reposetup() was called.
$ hg init $TESTTMP/cmdservelfs
$ cat >> $TESTTMP/cmdservelfs/.hg/hgrc << EOF
> [extensions]
> lfs =
> EOF
--------------------------------------------------------------------------------
Case #1: client with non-lfs content and the extension disabled; server with
non-lfs content, and the extension enabled.
$ cd client
$ echo 'non-lfs' > nonlfs.txt
......@@ -58,10 +77,63 @@
--------------------------------------------------------------------------------
Case #1: client with non-lfs content and the extension disabled; server with
non-lfs content, and the extension enabled.
$ cd client
$ echo 'non-lfs' > nonlfs.txt
$ hg ci -Aqm 'non-lfs'
>>> from __future__ import absolute_import
>>> from hgclient import check, readchannel, runcommand
>>> @check
... def diff(server):
... readchannel(server)
... # run an arbitrary command in the repo with the extension loaded
... runcommand(server, ['id', '-R', '../cmdservelfs'])
... # now run a command in a repo without the extension to ensure that
... # files are added safely..
... runcommand(server, ['ci', '-Aqm', 'non-lfs'])
... # .. and that scmutil.prefetchfiles() safely no-ops..
... runcommand(server, ['diff', '-r', '.~1'])
... # .. and that debugupgraderepo safely no-ops.
... runcommand(server, ['debugupgraderepo', '-q', '--run'])
*** runcommand id -R ../cmdservelfs
000000000000 tip
*** runcommand ci -Aqm non-lfs
*** runcommand diff -r .~1
diff -r 000000000000 nonlfs.txt
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/nonlfs.txt Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,1 @@
+non-lfs
*** runcommand debugupgraderepo -q --run
upgrade will perform the following actions:
requirements
preserved: dotencode, fncache, generaldelta, revlogv1, store
beginning upgrade...
repository locked and read-only
creating temporary repository to stage migrated data: * (glob)
(it is safe to interrupt this process any time before data migration completes)
migrating 3 total revisions (1 in filelogs, 1 in manifests, 1 in changelog)
migrating 132 bytes in store; 129 bytes tracked data
migrating 1 filelogs containing 1 revisions (9 bytes in store; 8 bytes tracked data)
finished migrating 1 filelog revisions across 1 filelogs; change in size: 0 bytes
migrating 1 manifests containing 1 revisions (53 bytes in store; 52 bytes tracked data)
finished migrating 1 manifest revisions across 1 manifests; change in size: 0 bytes
migrating changelog containing 1 revisions (70 bytes in store; 69 bytes tracked data)
finished migrating 1 changelog revisions; change in size: 0 bytes
finished migrating 3 total revisions; total change in store size: 0 bytes
copying phaseroots
data fully migrated to temporary repository
marking source repository as being upgraded; clients will be unable to read from repository
starting in-place swap of repository data
replaced files will be backed up at * (glob)
replacing store...
store replacement complete; repository was inconsistent for *s (glob)
finalizing requirements file and making repository readable again
removing temporary repository * (glob)
copy of old repository backed up at * (glob)
the old repository will not be deleted; remove it to free up disk space once the upgraded repository is verified
$ grep 'lfs' .hg/requires $SERVER_REQUIRES
[1]
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment