Skip to content
Snippets Groups Projects
  1. Apr 15, 2020
    • Pierre-Yves David's avatar
      upgrade: support the --quiet flag · e295ba238bd8
      Pierre-Yves David authored
      The command is currently very verbose with a various bit of output being time
      sensitive or randomized. The make invocation bulky and hard to match in the
      test. We move various message from `ui.write` to `ui.status` in order for the
      `--quiet` flag to have effect and helps the situation.
      
      As a side benefit, we can replace the various redirection to > /dev/null with the --quiet flag.
      
      Differential Revision: https://phab.mercurial-scm.org/D8429
      e295ba238bd8
    • Pierre-Yves David's avatar
      upgrade: clearly list optimisations · 5ee4b2119af9
      Pierre-Yves David authored
      This makes the command operation clearer. And this will be necessary to have
      a short version of this information with the next changesets, teaching `hg
      debugupgraderepo` about `--quiet`.
      
      Differential Revision: https://phab.mercurial-scm.org/D8428
      5ee4b2119af9
  2. Apr 14, 2020
  3. Apr 05, 2020
    • Pierre-Yves David's avatar
      nodemap: also warm manifest nodemap with other caches · 97ebdb192b00
      Pierre-Yves David authored
      The `hg debugupdatecache` command now also warm the persistent nodemap for the
      manifest (when applicable).
      
      Differential Revision: https://phab.mercurial-scm.org/D8411
      97ebdb192b00
    • Pierre-Yves David's avatar
      nodemap: also use persistent nodemap for manifest · 640d5b3bd060
      Pierre-Yves David authored
      The manifest as a different usage pattern than the changelog. First, while the
      lookup in changelog are not garanteed to match, the lookup in the manifest
      nodemap come from changelog and will exist in the manifest. In addition, looking
      up a manifest almost always result in unpacking a manifest an operation that
      rarely come cheap.
      
      Nevertheless, using a persistent nodemap provide a significant gain for some
      operations.
      
      
      For our measurementw, we use `hg cat --rev REV FILE` on the our reference
      mozilla-try. On this repository the persistent nodemap cache is about 29 MB in
      side for a total store side of 11,988 MB
      
      File with large history (file: b2g/config/gaia.json, revision: 195a1146daa0)
      
          no optimisation:                             0.358s
          using mmap for index:                        0.297s (-0.061s)
          persistent nodemap for changelog only:       0.275s (-0.024s)
          persistent nodemap for manifest too:         0.258s (-0.017s)
      
      File with small history (file: .hgignore, revision: 195a1146daa0)
      
          no optimisation:                             0.377s
          using mmap for index:                        0.296s (-0.061s)
          persistent nodemap for changelog only:       0.274s (-0.022s)
          persistent nodemap for manifest too:         0.257s (-0.017s)
      
      Same file but using a revision (8ba995b74e18) with a smaller manifest (3944829
      bytes vs  10 bytes)
      
          no optimisation:                             0.192s (-0.185s)
          using mmap for index:                        0.131s (-0.061s)
          persistent nodemap for changelog only:       0.106s (-0.025s)
          persistent nodemap for manifest too:         0.087s (-0.019s)
      
      Differential Revision: https://phab.mercurial-scm.org/D8410
      640d5b3bd060
    • Pierre-Yves David's avatar
      nodemap: create files in the repository used in the test · 2c073d82fbfa
      Pierre-Yves David authored
      We need a manifest with more content to test persistent nodemap for manifest.
      This change the repository content and affect all the hashes.
      
      Differential Revision: https://phab.mercurial-scm.org/D8409
      2c073d82fbfa
  4. May 07, 2020
  5. May 04, 2020
  6. May 01, 2020
  7. Mar 05, 2020
    • Pierre-Yves David's avatar
      copies: fix the changeset based algorithm regarding merge · 45f3f35cefe7
      Pierre-Yves David authored
      In 99ebde4fec99, we changed the list of files stored into the `files` field.
      This lead to the changeset centric copy algorithm to break in various merge
      situation involving merge. Older information could reach the merge through
      `p1`, and while information from `p2` was strictly fresher, it would get
      overwritten anyway.
      
      We update the situation with more details about which revision introduces rename
      information. This help use making the right decision in case of merge.
      
      We are now running a more comprehensive suite of test with include this kind of
      situation. The behavior differ slightly from the filelog based in a couple of
      instance. There is mostly two distinct cases:
      
      1) there are conflicting rename information in a merge (different rename history
      on each side). In this case the filelog based implementation arbitrarily pick a
      side based on the file-revision-number. So it depends on a local factor. The
      changeset centric algorithm will use a deterministic approach, by picking the
      information coming from the first parent of the merge. This is stable across
      different clone.
      
      2) rename information related to file that exist in both source and destination.
      The filelog based implementation do not even try to detect these, however the
      changeset centric one get them for "free" (it is simpler to detect them than
      not).
      
      The new implementation focus on correctness. Performance improvement will come
      later.
      
      Differential Revision: https://phab.mercurial-scm.org/D8244
      45f3f35cefe7
  8. May 01, 2020
  9. Apr 16, 2020
  10. Apr 24, 2020
    • Gregory Szorc's avatar
      automation: support building Python 3 MSI installers · 5e788dc7fb5d
      Gregory Szorc authored
      This is very similar to what we just did for Inno.
      
      Differential Revision: https://phab.mercurial-scm.org/D8484
      5e788dc7fb5d
    • Gregory Szorc's avatar
      automation: support building Python 3 Inno installers · 802ee93c205d
      Gregory Szorc authored
      The core packaging code now supports building Python 3 installers
      using PyOxidizer. Let's teach the automation code to invoke it so
      that we produce both Python 2 and Python 3 based exe installers.
      
      When publishing the artifacts, the Python 3 versions are preferred
      over the Python 2 versions given their higher weight (10 versus 9).
      This may be a controversial change. But I think making Python 3
      the default is warranted, as it is the future. The Python 2 installers
      are still fully supported and can be installed should issues with
      Python 3 arise.
      
      Differential Revision: https://phab.mercurial-scm.org/D8483
      802ee93c205d
    • Gregory Szorc's avatar
      automation: add extra arguments when building Inno · 47609da15379
      Gregory Szorc authored
      These were being fed into the template expansion but not being
      used. This meant --version was not getting set when it should have
      been.
      
      Differential Revision: https://phab.mercurial-scm.org/D8482
      47609da15379
    • Gregory Szorc's avatar
      packaging: add -python2 to Windows installer filenames · 9ade217b550d
      Gregory Szorc authored
      We just taught the Windows installers to produce Python 3 variants
      built with PyOxidizer.
      
      Our plan is to publish both Python 2 and Python 3 versions of the
      installers for Mercurial 5.4.
      
      This commit teaches the Inno and WiX installers to add an optional
      string suffix to the installer name. On Python 2, that suffix is
      "-python2." We reserve the existing name for the Python 3 installers,
      which we want to make the default.
      
      Differential Revision: https://phab.mercurial-scm.org/D8479
      9ade217b550d
    • Gregory Szorc's avatar
      automation: support building Windows wheels for Python 3.7 and 3.8 · 9d441f820c8b
      Gregory Szorc authored
      The time has come to support Python 3 on Windows.
      
      Let's teach our automation code to produce Windows wheels for
      Python 3.7 and 3.8.
      
      We could theoretically support 3.5 and 3.6. But I don't think
      it is worth it. People on Windows generally use the Mercurial
      installers, not wheels. And I'd prefer we limit variability
      and not have to worry about supporting earlier Python versions
      if it can be helped.
      
      As part of this, we change the invocation of pip to `python.exe -m pip`,
      as this is what is being recommended in Python docs these days. And
      it seemed to be required to avoid a weird build error. Why, I'm not
      sure. But it looks like pip was having trouble finding a Visual Studio
      files when invoked as `pip.exe` but not when using `python.exe -m pip`.
      Who knows.
      
      Differential Revision: https://phab.mercurial-scm.org/D8478
      9d441f820c8b
  11. Apr 21, 2020
    • Gregory Szorc's avatar
      packaging: support building WiX installers with PyOxidizer · 234882d17814
      Gregory Szorc authored
      We initially implemented PyOxidizer support for Inno installers.
      That did most of the heavy work of integrating PyOxidizer into
      the packaging system. Implementing WiX installer support was
      pretty straightforward.
      
      Aspects of this patch look very similar to Inno's.
      
      The main difference is the handling of the Visual C++
      Redistributable Runtime files.
      
      The WiX installer was formerly using merge modules to
      install the VC++ 9.0 runtime because this feature is
      supported by the WiX installer (it isn't easily available
      to Inno installers).
      
      Our strategy for the runtime files is to install the
      vcruntime140.dll file next to hg.exe just like any other
      file. While we could leverage WiX's functionality for invoking
      a VCRedist installer, I don't want to deal with the complexity
      at this juncture. So, we let run_pyoxidizer() copy vcruntime140.dll
      into the staging directory (like it does for Inno) and our
      dynamic WiX XML generator picks it up as a regular file and
      installs it.
      
      We did, however, have to teach mercurial.wxs how to conditionally
      use the merge modules. But this was rather straightforward.
      
      Comparing the file layout of the WiX installers before and
      after:
      
      * Various lib/*.{pyd, dll} files no longer exist
      * python27.dll was replaced by python37.dll
      * vcruntime140.dll was added
      
      All these changes are expected due to the transition to
      Python 3 and to PyOxidizer, which embeded the .pyd and .dll files
      in hg.exe.
      
      Differential Revision: https://phab.mercurial-scm.org/D8477
      234882d17814
    • Gregory Szorc's avatar
      packaging: move version derivation to run_wix_packaging() · a5740490eb5f
      Gregory Szorc authored
      With the previous commit moving signing inline, we no
      longer need to compute the version string in build_installer()
      and can instead move this logic to run_wix_packaging(). This
      makes the logic in build_installer() simpler, which makes it
      easier to implement alternate building mechanisms.
      
      Differential Revision: https://phab.mercurial-scm.org/D8476
      a5740490eb5f
    • Gregory Szorc's avatar
      packaging: integrate signing into run_wix_packaging() · a39984091380
      Gregory Szorc authored
      Previously, signing was implemented via a separate function
      which called build_installer() and then called signing
      functionality.
      
      In this model, in order to implement an alternative build
      mechanism, we would have to invent a new variant to handle
      signing as well.
      
      This commit merges the signing logic into the function invoking
      wix. If we pass an argument holding metadata about how to sign,
      we sign hg.exe and the installer. This means all we have to
      do is pass in signing info and the signing just works.
      
      A slight change here is that signing of hg.exe happens in the
      staging directory as opposed to before the staging directory
      is populated. I don't think this matters.
      
      Differential Revision: https://phab.mercurial-scm.org/D8475
      a39984091380
    • Gregory Szorc's avatar
      packaging: isolate invocation of WiX to own function · 92627c42e7c2
      Gregory Szorc authored
      Like we did for Inno, we want to split out the building
      of Mercurial from invoking the packaging tool so that we
      can introduce an alternate build mechanism.
      
      As part of this refactor, there are inconsequential changes
      to file layouts. Before, some shared files such as the
      WiX binaries and merge modules would be installed under
      build/. Now, they are installed under build/wix-*. This
      is to keep implementation simpler. But it also helps keep
      build state more isolated.
      
      Differential Revision: https://phab.mercurial-scm.org/D8474
      92627c42e7c2
  12. Apr 24, 2020
    • Gregory Szorc's avatar
      packaging: support building Inno installer with PyOxidizer · 94f4f2ec7dee
      Gregory Szorc authored
      We want to start distributing Mercurial on Python 3 on
      Windows. PyOxidizer will be our vehicle for achieving that.
      
      This commit implements basic support for producing Inno
      installers using PyOxidizer.
      
      While it is an eventual goal of PyOxidizer to produce
      installers, those features aren't yet implemented. So our
      strategy for producing Mercurial installers is similar to
      what we've been doing with py2exe: invoke a build system to
      produce files then stage those files into a directory so they
      can be turned into an installer.
      
      We had to make significant alterations to the pyoxidizer.bzl
      config file to get it to produce the files that we desire for
      a Windows install. This meant differentiating the build targets
      so we can target Windows specifically.
      
      We've added a new module to hgpackaging to deal with interacting
      with PyOxidizer. It is similar to pyexe: we invoke a build process
      then copy files to a staging directory. Ideally these extra
      files would be defined in pyoxidizer.bzl. But I don't think it
      is worth doing at this time, as PyOxidizer's config files are
      lacking some features to make this turnkey.
      
      The rest of the change is introducing a variant of the
      Inno installer code that invokes PyOxidizer instead of
      py2exe.
      
      Comparing the Python 2.7 based Inno installers with this
      one, the following changes were observed:
      
      * No lib/*.{pyd, dll} files
      * No Microsoft.VC90.CRT.manifest
      * No msvc{m,p,r}90.dll files
      * python27.dll replaced with python37.dll
      * Add vcruntime140.dll file
      
      The disappearance of the .pyd and .dll files is acceptable, as
      PyOxidizer has embedded these in hg.exe and loads them from
      memory.
      
      The disappearance of the *90* files is acceptable because those
      provide the Visual C++ 9 runtime, as required by Python 2.7.
      Similarly, the appearance of vcruntime140.dll is a requirement
      of Python 3.7.
      
      Differential Revision: https://phab.mercurial-scm.org/D8473
      94f4f2ec7dee
  13. Apr 19, 2020
  14. Apr 30, 2020
    • Elmar Bartel's avatar
      diff: re-establish linear runtime performance · e58422afbc74
      Elmar Bartel authored
      The previous method with sum() and list() creates a new list object
      for every hunk. Then sum() is used to flatten out this sequence of
      lists.  The sum() function is not "lazy", but creates a new list object
      for every "+" operation and so this code had quadratic runtime behaviour.
      e58422afbc74
  15. Apr 24, 2020
  16. Apr 23, 2020
  17. Apr 20, 2020
  18. Apr 11, 2020
Loading