Skip to content
Snippets Groups Projects
Commit 57830bd0 authored by Katsunori FUJIWARA's avatar Katsunori FUJIWARA
Browse files

scmutil: add file object wrapper class to check ambiguity at closing

In Mercurial source tree, opening a file in "a"/"a+" mode like below
doesn't specify atomictemp=True for vfs, and this avoids file stat
ambiguity check by atomictempfile.

  - writing changes out in revlog layer uses "a+" mode
  - truncation in repair.strip() uses "a" mode
  - truncation in transaction._playback() uses "a" mode

If steps below occurs at "the same time in sec", all of mtime, ctime
and size are same between (1) and (3).

  1. append data to revlog-style file (and close transaction)
  2. discard appended data by truncation (strip or rollback)
  3. append same size but different data to revlog-style file again

Therefore, cache validation doesn't work after (3) as expected.

This patch adds file object wrapper class checkambigatclosing to check
(and get rid of) ambiguity at closing. It is used by vfs in subsequent
patch.

This is a part of ExactCacheValidationPlan.

    https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan

BTW, checkambigatclosing is tested in test-filecache.py, even though
it doesn't use filecache itself, because filecache assumes that file
stat ambiguity never occurs (and there is no another test-*.py related
to filecache).
parent 0c40e64d
No related branches found
No related tags found
No related merge requests found
......@@ -1431,3 +1431,34 @@
return
self._queue.put(fh, block=True, timeout=None)
class checkambigatclosing(closewrapbase):
"""Proxy for a file object, to avoid ambiguity of file stat
See also util.filestat for detail about "ambiguity of file stat".
This proxy is useful only if the target file is guarded by any
lock (e.g. repo.lock or repo.wlock)
Do not instantiate outside of the vfs layer.
"""
def __init__(self, fh):
super(checkambigatclosing, self).__init__(fh)
object.__setattr__(self, '_oldstat', util.filestat(fh.name))
def _checkambig(self):
oldstat = self._oldstat
if oldstat.stat:
newstat = util.filestat(self._origfh.name)
if newstat.isambig(oldstat):
# stat of changed file is ambiguous to original one
advanced = (oldstat.stat.st_mtime + 1) & 0x7fffffff
os.utime(self._origfh.name, (advanced, advanced))
def __exit__(self, exc_type, exc_value, exc_tb):
self._origfh.__exit__(exc_type, exc_value, exc_tb)
self._checkambig()
def close(self):
self._origfh.close()
self._checkambig()
......@@ -179,6 +179,56 @@
print("* file y created")
print(repo.cached)
def antiambiguity():
filename = 'ambigcheck'
# try some times, because reproduction of ambiguity depends on
# "filesystem time"
for i in xrange(5):
fp = open(filename, 'w')
fp.write('FOO')
fp.close()
oldstat = os.stat(filename)
if oldstat.st_ctime != oldstat.st_mtime:
# subsequent changing never causes ambiguity
continue
repetition = 3
# repeat changing via checkambigatclosing, to examine whether
# st_mtime is advanced multiple times as expecetd
for i in xrange(repetition):
# explicit closing
fp = scmutil.checkambigatclosing(open(filename, 'a'))
fp.write('FOO')
fp.close()
# implicit closing by "with" statement
with scmutil.checkambigatclosing(open(filename, 'a')) as fp:
fp.write('BAR')
newstat = os.stat(filename)
if oldstat.st_ctime != newstat.st_ctime:
# timestamp ambiguity was naturally avoided while repetition
continue
# st_mtime should be advanced "repetition * 2" times, because
# all changes occured at same time (in sec)
expected = (oldstat.st_mtime + repetition * 2) & 0x7fffffff
if newstat.st_mtime != expected:
print("'newstat.st_mtime %s is not %s (as %s + %s * 2)" %
(newstat.st_mtime, expected, oldstat.st_mtime, repetition))
# no more examination is needed regardless of result
break
else:
# This platform seems too slow to examine anti-ambiguity
# of file timestamp (or test happened to be executed at
# bad timing). Exit silently in this case, because running
# on other faster platforms can detect problems
pass
print('basic:')
print()
basic(fakerepo())
......@@ -191,3 +241,7 @@
print('setbeforeget:')
print()
setbeforeget(fakerepo())
print()
print('antiambiguity:')
print()
antiambiguity()
......@@ -58,3 +58,6 @@
* file y created
creating
string from function
antiambiguity:
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment