Skip to content
Snippets Groups Projects
  1. Feb 03, 2020
  2. Jan 29, 2020
  3. Jan 30, 2020
    • Jan Alexander Steffens (heftig)'s avatar
      worker: Use buffered input from the pickle stream · cb52e619
      Jan Alexander Steffens (heftig) authored
      On Python 3, "pickle.load" will raise an exception ("_pickle.UnpicklingError:
      pickle data was truncated") when it gets a short read, i.e. it receives fewer
      bytes than it requested.
      
      On our build machine, Mercurial seems to frequently hit this problem while
      updating a mozilla-central clone iff it gets scheduled in batch mode. It is easy
      to trigger with:
      
          #wipe the workdir
          rm -rf *
          hg update null
      
          chrt -b 0 hg update default
      
      I've also written the following program, which demonstrates the core problem:
      
          from __future__ import print_function
      
          import io
          import os
          import pickle
          import time
      
          obj = {"a": 1, "b": 2}
          obj_data = pickle.dumps(obj)
          assert len(obj_data) > 10
      
          rfd, wfd = os.pipe()
      
          pid = os.fork()
          if pid == 0:
              os.close(rfd)
      
              for _ in range(4):
                  time.sleep(0.5)
                  print("First write")
                  os.write(wfd, obj_data[:10])
      
                  time.sleep(0.5)
                  print("Second write")
                  os.write(wfd, obj_data[10:])
      
              os._exit(0)
      
          try:
              os.close(wfd)
      
              rfile = os.fdopen(rfd, "rb", 0)
      
              print("Reading")
              while True:
                  try:
                      obj_copy = pickle.load(rfile)
                      assert obj == obj_copy
                  except EOFError:
                      break
              print("Success")
          finally:
              os.kill(pid, 15)
      
      The program reliably fails with Python 3.8 and succeeds with Python 2.7.
      
      Providing the unpickler with a buffered reader fixes the issue, so let
      "os.fdopen" create one.
      
      https://bugzilla.mozilla.org/show_bug.cgi?id=1604486
      
      Differential Revision: https://phab.mercurial-scm.org/D8051
      cb52e619
  4. Feb 01, 2020
  5. Feb 02, 2020
  6. Feb 01, 2020
    • Matt Harbison's avatar
      packaging: bundle dulwich, keyring, and pywin32-ctypes with WiX too · 481caa4a
      Matt Harbison authored
      TortoiseHg installs these, which is possibly where they originated (though I
      would have thought it more likely to be in the WiX installer, given its
      heritage).  When I was working on the TortoiseHg app for Mac (which uses the
      similar `py2app`), it wasn't possible to use the keyring extension (even
      externally) without bundling this keyring package into the app.  Assuming the
      same principle applies here, these would enable some common extensions.  One of
      the things that the TortoiseHg packager on macOS does now is it adds the user's
      local `site-packages` directory to `sys.path`.  That would allow the user to
      install these critical modules in cases like this.  But that can probably wait
      for py3 packaging.
      
      The only difference in the installed packages that I see now is WiX also bundles
      distutils for some reason.  I suppose that's not harming anything, so I'm not
      touching it.
      
      The only orphans in the install directories when comparing WiX and Inno now is
      the Copying.txt vs COPYING.rtf, the two uninstaller files for Inno, and a
      `Mercurial.url` file in Inno.  I have no idea what that is, and it has *.ini
      syntax with a single field pointing to the Mercurial homepage.
      
      Differential Revision: https://phab.mercurial-scm.org/D8062
      481caa4a
    • Matt Harbison's avatar
      packaging: bundle the default mercurial.ini template with Inno also · a8786727
      Matt Harbison authored
      This is a step towards converging on the same installer content on Windows.
      
      Differential Revision: https://phab.mercurial-scm.org/D8061
      a8786727
    • Matt Harbison's avatar
      packaging: set the FileVersion field in the Inno installer executable · f37971c3
      Matt Harbison authored
      Previously, Properties > Details > File version showed "0.0.0.0".  This appears
      to be a longstanding issue, and not part of the refactoring this cycle.
      
      Differential Revision: https://phab.mercurial-scm.org/D8060
      f37971c3
    • Matt Harbison's avatar
      packaging: move the version normalization function to the util module · a70108a3
      Matt Harbison authored
      This will be used with Inno as well.  Since this module isn't platform specific,
      rename to include that this is meant for Windows.  (Mac has a different format.)
      
      Differential Revision: https://phab.mercurial-scm.org/D8059
      a70108a3
    • Matt Harbison's avatar
      resourceutil: account for the non-resource-like file hierarchy under py2exe · aab70b54
      Matt Harbison authored
      After 9e367157a990, config files for py2exe were expected to be in
      C:\Program Files\Mercurial\mercurial\defaultrc because of the implied resource
      structure of 'mercurial.defaultrc.*.rc', relative to the executable.
      Accomodating this would require changes to the WIX and Inno scripts (and perhaps
      the script that generates the WIX script), as well as 3rd party bundlers like
      TortoiseHg.  But these files aren't read as resources anyway- they fall back to
      the filesystem APIs.  (If we really wanted to carry on the charade, the
      installer would have to also sprinkle various empty __init__.py files around.)
      
      Instead, this simply prunes the 'mercurial.' portion of the resource name when
      run with py2exe.  (PyOxidizer uses the resources API, not the filesystem
      fallback, so it is unaffected.)  Since this hack only affects the py2 Windows
      installers and is less risky, I think it's reasonable.  We haven't needed to
      load any 3rd party resource up to this point, and would have to make packaging
      changes anyway to handle that.
      
      Differential Revision: https://phab.mercurial-scm.org/D8058
      aab70b54
  7. Jan 31, 2020
    • Matt Harbison's avatar
      wix: restore COPYING.rtf · f010a80e
      Matt Harbison authored
      This got truncated to 0 bytes in 0ab651b5f77c when the Phabricator extension
      crashed because it's a binary file.  That caused the license page in the WIX
      installer to be empty.  I don't remember if I needed to resubmit after the bug
      was fixed, so let's try this again with the current stable.  If this fails, I'll
      retry with 5.1 to see if this is a regression in the API changeover last cycle.
      
      Differential Revision: https://phab.mercurial-scm.org/D8052
      f010a80e
    • Matt Harbison's avatar
      resourceutil: correct the root path for file based lookup under py2exe · 9e367157
      Matt Harbison authored
      This silly copy/paste error caused "Mercurial" to be truncated from
      "C:\Program Files".  The fact that "helptext" and "defaultrc" are now in a
      subpackage of "mercurial" added it back on, and everything seemed to work.  But
      that broke if not installed to the default directory, and also caused TortoiseHg
      to look at Mercurial's config files instead of its own.
      
      Differential Revision: https://phab.mercurial-scm.org/D8054
      9e367157
  8. Jan 25, 2020
  9. Jan 23, 2020
  10. Jan 24, 2020
  11. Jan 22, 2020
  12. Jan 21, 2020
  13. Jan 09, 2020
  14. Jan 03, 2020
    • Kyle Lippincott's avatar
      progress: flush stderr after clearing · 4e0a6d15
      Kyle Lippincott authored
      On python3, ui.stderr is buffered, it seems, so we need to flush it to actually
      get the progress bar off the screen. This is important since ui.write() will
      call into progbar.clear() if it thinks there's a progress bar on the screen,
      with the intent that the next thing it outputs is at the beginning of the line
      (instead of at the end of the progress bar line). Without the flush, we buffer
      up the clearing of the screen, and we get some really weird/corrupt output.
      
      Differential Revision: https://phab.mercurial-scm.org/D7784
      4e0a6d15
  15. Dec 24, 2019
    • Matt Harbison's avatar
      subrepo: fix a crash when archiving an svn or git subrepo · 7ca8aa88
      Matt Harbison authored
      Only hgsubrepos have a repository attribute.  This is pretty hacky, but probably
      the best we can do on stable.  Pushing the lfstatus check down into the wrapper
      for hgsubrepo (and dropping the check for lfstatus at the top of
      `hgsubrepoarchive()`) resulted in various test failures because:
      
        1) hgsubrepoarchive isn't returning the number of files archived at the
           bottom, resulting in an error about += NoneType
        2) These copypasta archive wrappers don't use progress bars
        3) Largefiles are *not* currently archived when using extdiff (68822b7cdd01),
           but pushing this context manager down into the subrepo resulted in it
           apparently doing so (as evidenced by progress bars being dropped)
      
      The other uses of `lfstatus()` are not in the substate processing loop, so they
      shouldn't be an issue.
      
      I initially put testcases in this test for largefiles-{on,off}, and it flagged
      a bunch of exit code differences for `cat` and `diff`, so I backed that off.
      
      Differential Revision: https://phab.mercurial-scm.org/D7714
      7ca8aa88
  16. Jan 17, 2020
    • Matt Harbison's avatar
      phabricator: use .arcconfig for `phabricator.url` if not set locally · ff396501
      Matt Harbison authored
      This setting is also per repo; see the previous commit for details.
      
      The existing `conduit_uri` setting is the previous name of `phabricator.uri`[1]
      and while it could easily be queried before the latter for compatibility, the
      config in this repo has '/api' appended. That's already done in `callconduit()`,
      which would clearly end up giving the wrong result. It looks like the path of
      the URL is now ignored in user configs[2], so add the modern setting without it
      to this repo's .arcconfig.
      
      Sadly, we still need to have contributors configure `auth.hg.phabtoken` (and
      therefore `auth.hg.prefix` to link it to `phabricator.url`) in order to submit
      patches, but at least now it's localized to a single section.
      
      [1] https://secure.phabricator.com/book/phabricator/article/arcanist_new_project/
      [2] https://github.com/phacility/arcanist/blob/cc850163f30c4697e925df0d6212469679600a2c/scripts/arcanist.php#L271
      
      Differential Revision: https://phab.mercu...
      ff396501
    • Matt Harbison's avatar
      phabricator: use .arcconfig for the callsign if not set locally (issue6243) · 59b3fe1e
      Matt Harbison authored
      This makes things easier for people working with more than one repository
      because this file can be committed to each repository.  The bug report asks to
      read <repo>/.arcrc, but AFAICT, that file lives in ~/ and holds the credentials.
      And we already track an .arcconfig file.  Any callsign set globally is still
      used if that is all that is present, but .arcconfig will override it if
      available.  The idea behind letting the local hgrc override .arcconfig is that
      the developer may need to do testing against another server, and not dirty the
      working directory.
      
      Originally I was going to just try to read the callsign in `getrepophid()` if it
      wasn't present in the hg config.  That works fine, but I think it also makes
      sense to read the URL from this file too.  That would have worked less well
      because `readurltoken()` doesn't have access to the repo object to know where to
      find the file.  Supplimenting the config mechanism is less magical because it
      reports the source and value of the properties used, and it doesn't need to read
      the file twice.
      
      Invalid hgrc files generally cause the program to abort.  I only flagged it as a
      warning here because it's not our config file, not crucial to the whole program
      operating, and really shouldn't be corrupt in the typical case where it is
      checked into the repo.
      
      Differential Revision: https://phab.mercurial-scm.org/D7934
      59b3fe1e
    • Matt Harbison's avatar
      config: add a function to insert non-file based, but overridable settings · e2278581
      Matt Harbison authored
      This will be used in the next patch.
      
      Until relatively recently (473510bf0575), there was no official way for
      extensions to inject per-repo config data, so it probably makes sense that
      `ui.setconfig()` items are sticky, and not affected by loading more config
      files.  But that makes it cumbersome if the extension wants to allow the data it
      might add to be overridden by any data in the local hgrc file.  The only thing I
      could get to work was to load the local hgrc first, and then check if the source
      for the config item that should be overridden was *not* the local hgrc file
      name.  But that's brittle because in addition to the file name, the source
      contains the line number, there are the usual '\' vs '/' platform differences,
      etc.
      
      Differential Revision: https://phab.mercurial-scm.org/D7933
      e2278581
    • Matt Harbison's avatar
      tests: restore phabricator tests and regenerate the recordings · a5e3f384
      Matt Harbison authored
      These contain the new API chatter.  Most of the changes are because some new
      commits were created, but they're pretty obviously equivalent.  I have no idea
      why the last recording contains real data, whereas it previously looked fake.
      
      Differential Revision: https://phab.mercurial-scm.org/D7920
      a5e3f384
  17. Jan 07, 2020
  18. Jan 18, 2020
Loading