Skip to content
Snippets Groups Projects
  1. Nov 08, 2024
  2. Oct 28, 2024
  3. Nov 08, 2024
    • Pierre-Yves David's avatar
      ci: rationalize variable usage · b3d6afa6
      Pierre-Yves David authored
      The usage of "extends" allow to skip a lot of duplication. We also introduce
      more fine grained variables to help finer override.
      b3d6afa6
    • Pierre-Yves David's avatar
      ci: move some variables closer to their usage · 74a9fbc1
      Pierre-Yves David authored
      These were defined globaly because we had trouble making them inherited. This
      is now fixed, so we can get them were they belong.
      74a9fbc1
    • Pierre-Yves David's avatar
      ci: use extends instead of <<: *x · 75bc7557
      Pierre-Yves David authored
      The old form is a yaml construct that make it hard to share variable definition.
      
      The "extends:" key is a gitlab specific that preserve the variable definition
      and just add the new ones. This will help us to reduce duplication.
      
      This has the effect of fixing some of variants definition we though we set while
      we did actually not. Most notably, the "rust" variant for 3.12 and 3.13 seems
      fully broken in the CI (possibly because some rust-cpython version issue?).
      
      This changeset only reveal such breackage and does not introduce them.
      75bc7557
  4. Nov 07, 2024
  5. Oct 15, 2024
  6. Nov 08, 2024
  7. Nov 07, 2024
  8. Nov 08, 2024
  9. Nov 06, 2024
  10. Nov 07, 2024
  11. Nov 05, 2024
    • Matt Harbison's avatar
      tests: bump the wait timeouts in test-racy-mutations.t · 16b300e3
      Matt Harbison authored
      This was done to try to figure out why there's output differences in the
      previous two commits- low timeouts have been a cause of a lot of problems on
      Windows.  That doesn't seem to be the case here, but I'm leaving it in anyway
      to avoid sporadic failures.
      16b300e3
    • Matt Harbison's avatar
      tests: add a "missing" tests for manifest content in test-racy-mutations.t · bda065f3
      Matt Harbison authored
      Trying to figure out why the divergence in behavior on Windows.  The first test
      shows everything is the same on all platforms; the second shows that the other
      #testcase also diverges.  The difference might be that `00manifest.i` doesn't
      get updated (and copied over) after `pre-race` is created, but I've no idea why
      that would be either.
      bda065f3
    • Matt Harbison's avatar
      tests: conditionalize missing output in test-racy-mutations.t on Windows · 54be3c9b
      Matt Harbison authored
      No idea why this is only missing on Windows.  I verified that the line marked
      as missing directly above this for the changelog is, in fact, missing on both
      Windows and Linux.  So there's probably work to do in this area on all
      platforms.  It would be nice to figure out what is going on, but this appeases
      CI in the meantime.
      54be3c9b
  12. Nov 07, 2024
  13. Nov 05, 2024
  14. Nov 04, 2024
    • Matt Harbison's avatar
      streamclone: disable the volatile file open handle optimization on Windows · e4b242f9
      Matt Harbison authored
      Leaving files open caused new failures like this, since a47f09da8bd1:
      
          diff --git a/tests/test-persistent-nodemap-stream-clone.t b/tests/test-persistent-nodemap-stream-clone.t
          --- a/tests/test-persistent-nodemap-stream-clone.t
          +++ b/tests/test-persistent-nodemap-stream-clone.t
          @@ -115,7 +115,12 @@ Do a mix of clone and commit at the same
             $ (hg clone -U --stream ssh://user@dummy/test-repo stream-clone-race-1 --debug 2>> clone-output | grep -E '00(changelog|manifest)' >> clone-output; touch $HG_TEST_STREAM_WALKED_FILE_3) &
             $ $RUNTESTDIR/testlib/wait-on-file 10 $HG_TEST_STREAM_WALKED_FILE_1
             $ hg -R test-repo/ commit -m foo
          -  created new head
          +  transaction abort!
          +  failed to recover 00changelog.n ([WinError 32] The process cannot access the file because it is being used by another process: b'$STR_REPR_TESTTMP\\test-repo/.hg/store/00changelog.n' -> b'$STR_REPR_TESTTMP\\test-repo/.hg/store/00changelog.n-f418dcd6')
          +  rollback failed - please run hg recover
          +  (failure reason: [WinError 32] The process cannot access the file because it is being used by another process: b'$STR_REPR_TESTTMP\\test-repo/.hg/store/00changelog.n' -> b'$STR_REPR_TESTTMP\\test-repo/.hg/store/00changelog.n-f418dcd6')
          +  abort: The process cannot access the file because it is being used by another process: '$TESTTMP\test-repo\.hg\store\00changelog.n'
          +  [255]
             $ touch $HG_TEST_STREAM_WALKED_FILE_2
             $ $RUNTESTDIR/testlib/wait-on-file 10 $HG_TEST_STREAM_WALKED_FILE_3
             $ cat clone-output
      
      Since the `VolatileManager` falls back to the old copy method when the open file
      threshold is exceeded, this just drops the threshold so that only 1 file is
      open.  The actual value used (2) is unexpected, and explained inline.  I'd like
      to have a config option for this so that we can test both ways (in theory, it
      could resort to copies on non-Windows systems too), but I don't see a `uimod.ui`
      handy.
      
      Alternately, I tried replacing the 3 `open()` calls in the `VolatileManager`
      with `util.posixfile()`, but that simply hung the test on Windows for some
      reason, I think on the same line that's indicated as failing above.  (There was
      a `grep` command hanging around, as well as `hg  -R test-repo serve --stdio`.)
      e4b242f9
    • Matt Harbison's avatar
      tests: treat `select` as a built-in module on Windows · 3ba998d7
      Matt Harbison authored
      This fixes:
      
          --- C:/Users/Matt/hg/tests/test-check-module-imports.t
          +++ C:/Users/Matt/hg/tests/test-check-module-imports.t.err
          @@ -43,3 +43,15 @@
             > -X tests/test-verify-repo-operations.py \
             > -X tests/test-extension.t \
             > | sed 's-\\-/-g' | "$PYTHON" "$import_checker" -
          +  hgext/zeroconf/Zeroconf.py:86: stdlib import "socket" follows local import: select\r (esc)
          +  hgext/zeroconf/Zeroconf.py:87: stdlib import "struct" follows local import: select\r (esc)
          +  hgext/zeroconf/Zeroconf.py:88: stdlib import "threading" follows local import: select\r (esc)
          +  hgext/zeroconf/Zeroconf.py:89: stdlib import "time" follows local import: select\r (esc)
          +  hgext/zeroconf/Zeroconf.py:90: stdlib import "traceback" follows local import: select\r (esc)
          +  mercurial/posix.py:18: stdlib import "stat" follows local import: select\r (esc)
          +  mercurial/posix.py:19: stdlib import "sys" follows local import: select\r (esc)
          +  mercurial/posix.py:20: stdlib import "tempfile" follows local import: select\r (esc)
          +  mercurial/posix.py:21: stdlib import "typing" follows local import: select\r (esc)
          +  tests/tinyproxy.py:19: stdlib import "socket" follows local import: select\r (esc)
          +  tests/tinyproxy.py:20: stdlib import "sys" follows local import: select\r (esc)
          +  [1]
      
          ERROR: test-check-module-imports.t output changed
      3ba998d7
  15. Oct 31, 2024
    • Matt Harbison's avatar
      hgweb: skip logging ConnectionAbortedError · 891f6d56
      Matt Harbison authored
      Not stacktracing on `ConnectionResetError` was added in 6bbb12cba5a8 (though it
      was spelled differently for py2 support), but for some reason Windows
      occasionally triggers a `ConnectionAbortedError` here across various *.t files
      (notably `test-archive.t` and `test-lfs-serve-access.t`, but there are others).
      The payload that fails to send seems to be the html that describes the error to
      the client, so I suspect some code is seeing the error status code and closing
      the connection before the server gets to write this html.  So don't log it, for
      test stability- nothing we can do anyway.
      
      FWIW, the CPython implementation of wsgihander specifically ignores these two
      errors, plus `BrokenPipeError`, with a comment that "we expect the client to
      close the connection abruptly from time to time"[1].  The `BrokenPipeError` is
      swallowed a level up in `do_write()`, and avoids writing the response following
      this stacktrace.  I'm puzzled why a response is being written after these
      connection errors are detected- the CPython code referenced doesn't, and the
      connection is now broken at this point.  Perhaps these errors should both be
      handled with the `BrokenPipeError` after the freeze.
      
      (The refactoring away from py2 compat may not be desireable in the freeze, but
      this is much easier to read, and obviously correct given the referenced CPython
      code.)
      
      I suspect this is what 6bceecb28806 was attempting to fix, but it wasn't
      specific about the sporadic errors it was seeing.
      
      [1] https://github.com/python/cpython/blob/b2eaa75b176e07730215d76d8dce4d63fb493391/Lib/wsgiref/handlers.py#L139
      891f6d56
  16. Oct 25, 2024
    • Matt Harbison's avatar
      ci: add a runner for Windows 10 · 8766d47e
      Matt Harbison authored
      This is currently only manually invoked, and allows for failure because we only
      have a single runner that takes over 2h for a full run, and there are a handful
      of flakey tests, plus 3 known failing tests.
      
      The system being used here is running MSYS, Python, Visual Studio, etc, as
      installed by `install-windows-dependencies.ps1`.  This script installs
      everything to a specific directory instead of using the defaults, so we adjust
      the MinGW shell path to compensate.  Additionally, the script doesn't install
      the launcher `py.exe`.  It is possible to adjust the script to install it, but
      it's an option to an existing python install (instead of a standalone installer),
      and I've had the whole python install fail and rollback when requested to install
      the launcher if it detects a newer one is already installed.  In short, it is
      a point of failure for a feature we don't (yet?) need.
      
      Unlike other systems where the intepreter name includes the version, everything
      here is `python.exe`, so they can't all exist on `PATH` and let the script
      choose the desired one.  (The `py.exe` launcher would accomplish, using the
      registry instead of `PATH`, but that wouldn't allow for venv installs.)  Because
      of this, switch to the absolute path of the python interpreter to be used (in
      this case a venv created from the py39 install, which is old, but what both
      pyoxidizer and TortoiseHg currently use).
      
      The `RUNTEST_ARGS` hardcodes `-j8` because this system has 4 cores, and
      therefore runs 4 parallel tests by default.  However on Windows, using more
      parallel tests than cores results in better performance for whatever reason.  I
      don't have an optimal value yet (ideally the runner itself can make the
      adjustment on Windows), but this results in saving ~15m on a full run that
      otherwise takes ~2.5h.  I'm also not concerned about how it would affect other
      Windows machines, because we don't have any at this point, and I have no idea
      when we can get more.
      
      As far as system setup goes, the CI is run by a dedicated user that lacks admin
      rights.  The install script was run by an admin user, and then the standard user
      was configured to use it.  If I set this up again, I'd probably give the
      dedicated user admin rights to run the install script, and reset to standard
      user rights when done.  The python intepreter failed in weird ways when run by
      the standard user until it was manually reinstalled by the standard user:
      
          Fatal Python error: init_fs_encoding: failed to get the Python codec of the
              filesystem encoding
      
      Additionally, changing the environment through the Windows UI prompts to
      escalate to an admin user, and then setting the user level environment variables
      like `TEMP` and `PATH` (to try to avoid exceeding the 260 character path limit)
      didn't actually change the user's environment.  (Likely it changed the admin
      user's environment, but I didn't confirm that.)  I ended up having to use the
      registry editor for the standard user to make those changes.
      8766d47e
  17. Oct 11, 2024
    • Matt Harbison's avatar
      tests: disable a section of `test-hgrc.t` that may hit a zeroconf bug · 74e16d8c
      Matt Harbison authored
      This effectively re-disables the same test as cce9e7d2fb92, but unconditionally
      because it's not a pyoxidizer-specific problem (see below and 997c9b2069d1).
      
      I can run the test locally fine, with the same venv as CI is using, and have had
      multiple CI runs that don't hit this.  But one failed with this:
      
          --- /private/tmp/mercurial-ci/tests/test-hgrc.t
          +++ /private/tmp/mercurial-ci/tests/test-hgrc.t.err
          @@ -305,5 +305,17 @@
             [255]
      
             $ HGRCSKIPREPO=1 hg paths --config extensions.zeroconf=
          +  Traceback (most recent call last):
          +    File "/private/tmp/hgtests.7idf706t/install/lib/python/hgext/zeroconf/Zeroconf.py", line 966, in run
          +      self.readers[sock].handle_read()
          +    File "/private/tmp/hgtests.7idf706t/install/lib/python/hgext/zeroconf/Zeroconf.py", line 1020, in handle_read
          +      msg = DNSIncoming(data)
          +    File "/private/tmp/hgtests.7idf706t/install/lib/python/hgext/zeroconf/Zeroconf.py", line 537, in __init__
          +      self.readOthers()
          +    File "/private/tmp/hgtests.7idf706t/install/lib/python/hgext/zeroconf/Zeroconf.py", line 650, in readOthers
          +      self.readCharacterString(),
          +    File "/private/tmp/hgtests.7idf706t/install/lib/python/hgext/zeroconf/Zeroconf.py", line 584, in readCharacterString
          +      length = ord(self.data[self.offset])
          +  TypeError: ord() expected string of length 1, but int found
             foo = $TESTTMP/bar
      
      The zeroconf extension has bytes vs str problems that are obvious from
      inspection alone, and nobody has complained, so I'm not going to let this block
      getting CI for macOS up and running.  Given that it's in the packet read code,
      I suspect that this 1) requires something on the network to speak mDNS, and 2)
      it is a timing issue if this is seen or not.  (The bytes vs str issue itself is
      real, but only happen if a response is received quickly.)
      74e16d8c
    • Matt Harbison's avatar
      tests: disable `test-git-interop.t` with a requirements directive · 6ca0771b
      Matt Harbison authored
      Note that the failures in this test affect all platforms.
      
      I don't like this, but the test has been broken for awhile because of dirstate
      API changes, and nobody noticed because the required `pygit2` package isn't
      installed on the CI systems.  I did install it on the mac CI system, which
      triggers this failure.  Disabling it is no worse than not running it due to the
      missing package, but at least this way the CI systems can get the package
      installed, and the test can be enabled and fixed eventually, without needing to
      alter the CI systems.
      
      The feature here is kind of abused.  I thought about adding one specifically to
      test for CI, but didn't feel like doing it at this point.  Maybe if we need to
      disable things to get the Windows CI off the ground (but that likely requires
      testing for CI + platform).
      6ca0771b
  18. Nov 01, 2024
    • Matt Harbison's avatar
      tests: stabilize `test-extdiff.t` on macOS · a81d2caf
      Matt Harbison authored
      The recent change in the extdiff extension to take into account whether the GUI
      is accessible in d1b54c152673 started triggering this.  I was able to run the
      test cleanly without this change at the console, but somewhere along the line, I
      read that the CI runner isn't able to access the GUI when not run as the root
      user.  This is causing CI failures, so we conditionalize these tests out where
      `DISPLAY` is set to a non empty value to force `procutil.isgui()` to be True,
      when it in fact doesn't have GUI access.
      a81d2caf
  19. Oct 27, 2024
  20. Oct 28, 2024
Loading