Skip to content
Snippets Groups Projects
  1. Apr 28, 2022
  2. Nov 03, 2021
  3. Oct 21, 2021
  4. Oct 20, 2021
  5. Oct 18, 2021
    • Kyle Lippincott's avatar
      pyoxidizer: update README.md with several small fixes · 16c3fe46929a
      Kyle Lippincott authored
      Currently, pyoxidizer.bzl does not mention the git commit that should be checked
      out, so these instructions are a bit difficult to follow right now (impossible,
      technically), so I removed the instruction to `git checkout <Git commit>` and
      the admonition to use a specific version of PyOxidizer. I don't even know if the
      project currently builds with the "0.7.0-pre" version that was previously
      recommended.
      
      As fallout from that change to not "pin" to a specific PyOxidizer, I had to
      update the Python version to use when running the tests.
      
      While here, I added a recommendation to use `--release`, as the primary reason
      for this project is performance, and it may have been leaving some on the table
      to not have that there.
      
      Differential Revision: https://phab.mercurial-scm.org/D11698
      16c3fe46929a
    • Kyle Lippincott's avatar
      pyoxidizer: disable using in-memory resources · c900d962e38b
      Kyle Lippincott authored
      It's possible that the errors are due to using an incompatible version of
      PyOxidizer; unfortunately the README.md file in this directory says to fetch a
      copy of PyOxidizer matching the commit in this pyoxidizer.bzl file, and yet the
      pyoxidizer.bzl file does not actually have a commit mentioned in it.
      
      By disabling in-memory modules, this appears to work on all platforms using the
      current head version of PyOxidizer, so let's disable them for now.
      
      Sample error (during `pyoxidizer build`):
      
      ```
      error[PYOXIDIZER_PYTHON_EXECUTABLE]: adding PythonExtensionModule<name=hgext.fsmonitor.pywatchman.bser>
      
      Caused by:
          extension module hgext.fsmonitor.pywatchman.bser cannot be loaded from memory but memory loading required
         --> ./pyoxidizer.bzl:140:5
          |
      140 |     exe.add_python_resources(exe.pip_install(["--verbose", ROOT]))
          |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ add_python_resources()
      
      
      error: adding PythonExtensionModule<name=hgext.fsmonitor.pywatchman.bser>
      
      Caused by:
          extension module hgext.fsmonitor.pywatchman.bser cannot be loaded from memory but memory loading required
      ```
      
      Differential Revision: https://phab.mercurial-scm.org/D11697
      c900d962e38b
  6. Aug 27, 2021
  7. Sep 17, 2021
  8. Aug 27, 2021
  9. Aug 26, 2021
  10. Aug 19, 2021
  11. Aug 18, 2021
  12. Jul 12, 2021
  13. May 06, 2021
    • Gregory Szorc's avatar
      pyoxidizer: use Python 3.9 (BC) · 159d2de3286c
      Gregory Szorc authored
      Newer versions of PyOxidizer use Python 3.9 by default. We
      previously pinned the version to 3.8 to facilitate porting to
      a new PyOxidizer version and diffing results.
      
      Now that the porting work is complete, let's bump Python
      to Python 3.9.
      
      This will effectively change our Windows Inno and WiX Python 3
      installers from Python 3.8 to 3.9.
      
      .. bc::
      
         Windows .msi and .exe installers now use Python 3.9 instead of
         Python 3.8.
      
      Differential Revision: https://phab.mercurial-scm.org/D10689
      159d2de3286c
    • Gregory Szorc's avatar
      pyoxidizer: support code signing · c8001d9c26f5
      Gregory Szorc authored
      Newer versions of PyOxidizer feature built-in support for
      code signing. You simply declare a code signer in the Starlark
      configuration file, activate it for automatic signing, and
      PyOxidizer will add code signatures to signable files as it
      encounters them.
      
      This commit teaches our Starlark configuration file to enable
      automatic code signing. But only on Windows for the moment, as our
      immediate goal is to overhaul the Windows packaging.
      
      The feature is opt-in: you must pass variables to PyOxidizer's
      build context via `pyoxidizer build --var` or
      `pyoxidizer build --var-env` to activate code signing.
      
      Differential Revision: https://phab.mercurial-scm.org/D10684
      c8001d9c26f5
    • Gregory Szorc's avatar
      pyoxidizer: support producing MSI installers · 603efb3845ba
      Gregory Szorc authored
      Newer versions of PyOxidizer have support for building WiX MSI
      installers "natively." Essentially, you can script the definition
      of your WiX installer via Starlark and PyOxidizer can invoke
      WiX tools to produce the installer.
      
      This commit teaches our PyOxidizer config file to produce MSI
      installers similarly to how
      `contrib/packaging/packging.py wix` would do it.
      
      We had to make a very minor change to `mercurial.wxs` to reflect
      different paths depending on who builds. This is because when
      PyOxidizer builds WiX installers, it does so from an isolated
      directory, not Mercurial's source directory. We simply copy the
      files into the build environment so they are accessible.
      
      After this change, running `pyoxidizer build msi` produces a nearly
      identical install layout to what the previous method produces.
      When I applied this series on top of the 5.8 tag, here is the
      list of differences and explanations:
      
      * docs/*.html files are missing from the new installer because the
        Python build environment doesn't have docutils.
      * .pyd and .exe files differ, likely because I'm using a different
        Visual Studio toolchain on my local computer than the official build
        environment.
      * Various .dist-info/ directories have different names. This is
        because older versions of PyOxidizer had buggy behavior and weren't
        properly normalizing package names in .dist-info/ directories. e.g.
        we went from `cached-property-1.5.2.dist-info` to
        `cached_property-1.5.2.dist-info`.
      * Translations (.mo files) may be missing if gettext isn't in %Path%.
        This is because the packaging.py code installs gettext and ensures
        it can be found.
      * Some *.dist-info/RECORD files vary due to SHA-256 content digest
        divergence due to build environment differences. (This should be
        harmless.)
      * The new install layout ships a python3.dll because newer versions
        of PyOxidizer ship this file.
      * The new install layout has a different vcruntime140.dll and also a
        vcruntime140_1.dll because newer versions of PyOxidizer ship a
        newer version of the Visual C++ Redistributable Runtime.
      
      The new PyOxidizer functionality is not yet integrated with
      packaging.py. This will come in a subsequent commit. So for now, the
      new functionality introduced here is unused.
      
      Differential Revision: https://phab.mercurial-scm.org/D10683
      603efb3845ba
    • Gregory Szorc's avatar
      pyoxidizer: use allocator_backend instead of raw_allocator · 3380848c6e04
      Gregory Szorc authored
      The name of this attribute changed in PyOxidizer 0.11.
      
      Differential Revision: https://phab.mercurial-scm.org/D10682
      3380848c6e04
    • Gregory Szorc's avatar
      pyoxidizer: simplify targets · 9438e9b7321a
      Gregory Szorc authored
      The split targets existed to enable the use of a non-default
      distribution flavor on Windows. Modern versions of PyOxidizer
      use the "standalone_dynamic" distribution flavor by default.
      So our split brain workaround is no longer needed.
      
      Here, we unify the targets. We also remove an unreferenced
      target function to create a resources file.
      
      Differential Revision: https://phab.mercurial-scm.org/D10681
      9438e9b7321a
    • Gregory Szorc's avatar
      pyoxidizer: pin Python to 3.8 · 95950dc832cf
      Gregory Szorc authored
      Newer versions of PyOxidizer use version 3.9 by default. As part of
      upgrading PyOxidizer, we want to pin the version at 3.8 so we can
      compare differences more easily.
      
      Differential Revision: https://phab.mercurial-scm.org/D10679
      95950dc832cf
    • Gregory Szorc's avatar
      pyoxidizer: remove some boilerplate in file · 3e18a29deec9
      Gregory Szorc authored
      This isn't used for anything and can safely be removed.
      
      Differential Revision: https://phab.mercurial-scm.org/D10678
      3e18a29deec9
  14. Mar 25, 2021
  15. Dec 03, 2020
  16. Nov 26, 2020
  17. Nov 19, 2020
  18. Nov 10, 2020
  19. Nov 03, 2020
  20. Oct 22, 2020
    • Gregory Szorc's avatar
      contrib: split Windows requirements into multiple files · d270b9b797a7
      Gregory Szorc authored
      Package support for Python 2 has diverged significantly. It is no
      longer trivial to maintain a single requirements file that supports
      both Python 2 and 3 because the set of packages and versions varies
      wildly.
      
      This commit split up the Windows requirements files so we have
      variants for Python 2 and 3. As part of this, I also renamed the
      files to have what I believe to be more reasonable naming ("win32"
      felt like a weird identifier to me).
      
      We can see that some package versions decreated on 2.7. This is
      because the old pinned versions weren't compatible with Python 2.
      d270b9b797a7
  21. Oct 19, 2020
  22. Oct 05, 2020
    • Gregory Szorc's avatar
      pyoxidizer: produce working Python 3 Windows installers (issue6366) · 57b5452a55d5
      Gregory Szorc authored
      While we've had code to produce Python 3 Windows installers with
      PyOxidizer, we haven't been advertising them on the web site due to
      a bug in making TLS connections and issues around resource handling.
      
      This commit upgrades our PyOxidizer install and configuration to
      use a recent Git commit of PyOxidizer. This new version of PyOxidizer
      contains a *ton* of changes, improvements, and bug fixes. Notably,
      Windows shared distributions now mostly "just work" and the TLS bug
      and random problems with Python extension modules in the standard
      library go away. And Python has been upgraded from 3.7 to 3.8.6.
      
      The price we pay for this upgrade is a ton of backwards incompatible
      changes to Starlark.
      
      I applied this commit (the overall series actually) on stable to
      produce Windows installers for Mercurial 5.5.2, which I published
      shortly before submitting this commit for review.
      
      In order to get the stable branch working, I decided to take a
      less aggressive approach to Python resource management. Previously,
      we were attempting to load all Python modules from memory and were
      performing some hacks to copy Mercurial's non-module resources
      into additional directories in Starlark. This commit implements
      a resource callback function in Starlark (a new feature since
      PyOxidizer 0.7) to dynamically assign standard library resources
      to in-memory loading and all other resources to filesystem loading.
      This means that Mercurial's files and all the other packages we ship
      in the Windows installers (e.g. certifi and pygments) are loaded
      from the filesystem instead of from memory. This avoids issues
      due to lack of __file__ and enables us to ship a working Python
      3 installer on Windows.
      
      The end state of the install layout after this patch is not
      ideal for @: we still copy resource files like templates and
      help text to directories next to the hg.exe executable. There
      is code in @ to use importlib.resources to load these files and
      we could likely remove these copies once this lands on @. But for
      now, the install layout mimics what we've shipped for seemingly
      forever and is backwards compatible. It allows us to achieve the
      milestone of working Python 3 Windows installers and gets us a
      giant step closer to deleting Python 2.
      
      Differential Revision: https://phab.mercurial-scm.org/D9148
      57b5452a55d5
  23. Oct 01, 2020
    • Martin von Zweigbergk's avatar
      rust: move rustfmt.toml to repo root so it can be used by `hg fix` · 426294d06ddc
      Martin von Zweigbergk authored
      `hg fix` runs the formatters from the repo root so it doesn't pick up
      the `rustfmt.toml` configs we had in each the `hg-core`, `hg-cpython`,
      and `rhg` packages, which resulted in warnings about `async fn` not
      existing in Rust 2015. This patch moves the `rustfmt.toml` file to the
      root so `hg fix` will use it.
      
      By putting the `rustfmt.toml` file in a higher-level directory, it
      also applies to the `chg` and `hgcli` packages. That makes
      `test-check-rust-format.t` fail, so this patch also applies the new
      formatting rules to those packages.
      
      Differential Revision: https://phab.mercurial-scm.org/D9142
      426294d06ddc
  24. May 12, 2020
  25. Apr 24, 2020
    • Gregory Szorc's avatar
      packaging: support building Inno installer with PyOxidizer · 94f4f2ec7dee
      Gregory Szorc authored
      We want to start distributing Mercurial on Python 3 on
      Windows. PyOxidizer will be our vehicle for achieving that.
      
      This commit implements basic support for producing Inno
      installers using PyOxidizer.
      
      While it is an eventual goal of PyOxidizer to produce
      installers, those features aren't yet implemented. So our
      strategy for producing Mercurial installers is similar to
      what we've been doing with py2exe: invoke a build system to
      produce files then stage those files into a directory so they
      can be turned into an installer.
      
      We had to make significant alterations to the pyoxidizer.bzl
      config file to get it to produce the files that we desire for
      a Windows install. This meant differentiating the build targets
      so we can target Windows specifically.
      
      We've added a new module to hgpackaging to deal with interacting
      with PyOxidizer. It is similar to pyexe: we invoke a build process
      then copy files to a staging directory. Ideally these extra
      files would be defined in pyoxidizer.bzl. But I don't think it
      is worth doing at this time, as PyOxidizer's config files are
      lacking some features to make this turnkey.
      
      The rest of the change is introducing a variant of the
      Inno installer code that invokes PyOxidizer instead of
      py2exe.
      
      Comparing the Python 2.7 based Inno installers with this
      one, the following changes were observed:
      
      * No lib/*.{pyd, dll} files
      * No Microsoft.VC90.CRT.manifest
      * No msvc{m,p,r}90.dll files
      * python27.dll replaced with python37.dll
      * Add vcruntime140.dll file
      
      The disappearance of the .pyd and .dll files is acceptable, as
      PyOxidizer has embedded these in hg.exe and loads them from
      memory.
      
      The disappearance of the *90* files is acceptable because those
      provide the Visual C++ 9 runtime, as required by Python 2.7.
      Similarly, the appearance of vcruntime140.dll is a requirement
      of Python 3.7.
      
      Differential Revision: https://phab.mercurial-scm.org/D8473
      94f4f2ec7dee
  26. Apr 16, 2020
  27. Apr 01, 2020
    • Gregory Szorc's avatar
      hgcli: customize for Mercurial · bc847878f4c0
      Gregory Szorc authored
      Now that we have a shiny new PyOxidizer-based hgcli project, let's
      customize it for Mercurial!
      
      This commit replaces the auto-generated pyoxidizer.bzl with one
      that installs Mercurial from the local source repository.
      
      A README.md with build instructions has been added.
      
      The Cargo.toml file has been updated to reflect the proper license
      and reference the added README.md.
      
      In my Linux environment, running the test suite yields 27 failures.
      
      It's worth noting the run time of the test harness on Linux on my
      Ryzen 3950X:
      
      before: 378s wall; 9982s user; 1195s sys
      after:  353s wall; 8996s user;  958s sys
      % orig: 93.4 wall;  90.1  user; 80.2 sys
      
      While I haven't measured explicitly, I suspect the performance win is
      due to in-memory resource loading (which is known to be faster than
      Python's filesystem importer).
      
      Differential Revision: https://phab.mercurial-scm.org/D8351
      bc847878f4c0
    • Gregory Szorc's avatar
      hgcli: add stub PyOxidizer project · af739894a4c1
      Gregory Szorc authored
      Using commit c772a1379c3026314eda1c8ea244b86c0658951d of
      PyOxidizer, I ran `pyoxidizer init-rust-project hgcli` to
      create a stub Rust project. The only modifications I made from
      what that command produced are:
      
      * Update location of pyembed crate to PyOxidizer's Git repository.
      * Removed some trailing whitespace from pyoxidizer.bzl
      * Added auto-generated Cargo.lock file
      
      Subsequent commits will modify the stub project to Mercurial's
      needs.
      
      Differential Revision: https://phab.mercurial-scm.org/D8350
      af739894a4c1
    • Gregory Szorc's avatar
      hgcli: remove legacy project · 02f66b23cba3
      Gregory Szorc authored
      This code is a logical precursor to PyOxidizer. It is now
      defunct.
      
      Differential Revision: https://phab.mercurial-scm.org/D8349
      02f66b23cba3
Loading