Python 2 removal extravaganza
As previously discussed in the last virtual sprint, on the mailing list and other discussions on IRC, the only thing holding back the removal of Python 2 compatibility from Mercurial is TortoiseHg Windows Python3 packaging.
For Mercurial itself, this means having a working Windows Python 3 CI. We already have Windows runners thanks to recent work by @gracinet, @mharbison72 and others, but we're still short of having a green CI run.
Tests that are known to fail on Windows Python 3 (fixes are in windows* topics, we will consolidate sometime soon):
-
test-convert-svn-sink.t
-
test-subrepo-svn.t
-
test-lfs-serve-access.t
-
test-profile.t
(a mix of shell incompatibility and real-time vs cpu time shenanigans) -
test-paths.t
(double-backslash-escaping issue) -
test-pager.t
(turns out the Windows behavior may be a good thing) -
test-http-proxy.t
(os.environ does uppercase normalization on windows, fixing this makes the test pass) -
test-transaction-rollback-on-revlog-split.t
(was a simple path normalization issue) -
test-transaction-rollback-on-sigpipe.t
(after some fixes by @mharbison72, was a matter of having an up-to-date bashturns out this is hell on earth, but we're making progress) -
test-bookflow.t
-
test-bookmarks.t
-
test-removeemptydirs.t
-
test-ssh-bundle1.t
-
test-ssh.t
: output changed
It is possible that these tests are flaky:
- test-pager.t (as see during this July 10th run)
- test-alias.t
- test-url-download.t
- test-hgweb-filelog.t (as see during this July 10th run)
- test-rebase-base-flag.t
- test-http.t (as see during this July 10th run
- test-archive.t (as see during this July 11th run)
- test-obsolete-divergent.t (see https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/234516 )
- test-shelve2.t#phasebased#abortcommand#continuecommand (see https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/234496)
- test-subrepo-recursion.t (see https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/234516 )
Tests not currently covered by CI that are known to fail on Windows py3:
- test-gendoc-da.t
- test-gendoc-de.t
- test-gendoc-ro.t
- test-gendoc-sv.t
After fixing the last critical bug (test-pager.t
, test-paths.t
), we can blacklist the rest and add the Windows CI to Heptapod to prevent further regressions, then go back to fixing the remaining issues.
Tests that are skipped locally for me (@mharbison72) on Windows that we should see if they can reasonably be made to run (modulo obvious stuff like symlinks, exec bits, and test-check-* that can be run on Linux). There are more than a handful that I disabled for Windows/MSYS after not being able to get to the bottom of them. There are also some tests that probably have too broad of an exclusion- e.g. why does test-rebase-inmemory.t
require executable bit support for the whole thing? (I probably shouldn't have dropped the symlink and exec bit tests from this list.)
- Skipped test-chg.t: missing feature: running with chg
- Skipped test-clone-cgi.t: system supports Windows with MSYS
- Skipped test-contrib-emacs.t: missing feature: GNU Emacs
- Skipped test-convert-bzr-*.t: missing feature: Breezy library and executable version >= 3.1
- Skipped test-convert-cvs-*.t: missing feature: cvs client/server
- Skipped test-convert-darcs.t: missing feature: darcs client
- Skipped test-convert-hg-svn.t: missing feature: subversion python bindings
- Skipped test-convert-mtn.t: missing feature: monotone client (>= 1.0)
- Skipped test-convert-p4*.t: missing feature: symbolic links
- Skipped test-convert-svn-*.t: missing feature: subversion python bindings
- Skipped test-dirstate-race.t: missing feature: Using the Rust extensions
- Skipped test-dirstate-race2.t: missing feature: Using the Rust extensions
- Skipped test-dirstate.t: missing feature: Using the Rust extensions
- Skipped test-filecache.py: missing feature: cacheable filesystem
- Skipped test-fuzz-targets.t: skipped
- Skipped test-git-interop.t: system supports Windows
- Skipped test-gpg.t: missing feature: gpg client
- Skipped test-hgignore.t: missing feature: Using the Rust extensions
- Skipped test-lock-badness.t: system supports Windows
- Skipped test-logtoprocess.t: system supports Windows
- Skipped test-newcgi.t: system supports Windows with MSYS
- Skipped test-newercgi.t: system supports Windows with MSYS
- Skipped test-nointerrupt.t: system supports Windows
- Skipped test-oldcgi.t: system supports Windows with MSYS
- Skipped test-purge.t: missing feature: Using the Rust extensions
- Skipped test-push-cgi.t: system supports Windows with MSYS
- Skipped test-rebase-inmemory.t: missing feature: executable bit
- Skipped test-remotefilelog-*.t: system supports Windows
- Skipped test-rhg.t: missing feature: running with rhg as 'hg'
- Skipped test-sparse-fsmonitor.t: skipped
- Skipped test-sparse-revlog.t: missing artifact, run "c:\Users\Matt\hg\tests/artifacts/scripts/generate-churning-bundle.py"
- Skipped test-sqlitestore.t: missing feature: sqlite3 module and matching cli is available
- Skipped test-status.t: missing feature: Using the Rust extensions
- Skipped test-verify-repo-operations.py: missing feature: allow slow tests (use --allow-slow-tests)
- Skipped test-wireproto-exchangev2-shallow.t: missing feature: sqlite3 module and matching cli is available
There may also be an issue with distutils requiring an out-of-date compiler with zstd, (got the error when trying to build python 2, so it may not matter in the end).
We need to update the base image with all the required dependencies and profile manipulation (mostly env variables) of the Heptapod Windows runner and clean-up heptapod-ci.yml
, then we should be good to go.
Once the CI is live, we can send a mail to the devel mailing-list to decide on the exact action for removing Python 2.