Skip to content
Snippets Groups Projects
  1. Mar 05, 2022
    • Martin von Zweigbergk's avatar
      contrib: add a partial-merge tool for sorted lists (such as Python imports) · 681b25ea579e
      Martin von Zweigbergk authored
      This is a pretty naive tool that uses a regular expression for
      matching lines. It is based on a Google-internal tool that worked in a
      similar way.
      
      For now, the regular expression is hard-coded to attempt to match
      single-line Python imports. The only commit I've found in the hg core
      repo where the tool helped was commit 9cd6292abfdf. I think that's
      because we often use multiple imports per import statement. I think
      this tool is still a decent first step (especially once the regex is
      made configurable in the next patch). The merging should ideally use a
      proper Python parser and do the merge at the AST (or CST?) level, but
      that's significantly harder, especially if you want to preserve
      comments and whitespace. It's also less generic.
      
      Differential Revision: https://phab.mercurial-scm.org/D12380
      681b25ea579e
  2. Mar 13, 2022
  3. Jan 18, 2021
    • Matt Harbison's avatar
      run-tests: work around the Windows firewall popup for server processes · 3e23794b9e1c
      Matt Harbison authored
      Windows doesn't have a `python3` executable, so cc0b332ab9fc attempted to work
      around the issue by copying the current python to `python3.exe`.  That put it in
      `_tmpbindir` because of failures in `test-run-tests.t` when using `_bindir`,
      which looked like a process was trying to open it to write out a copy while it
      was in use.  (Interestingly, I couldn't reproduce this running the test by
      itself in a loop for a couple of hours, but it happens constantly when running
      all tests.)  The problem with using `_tmpbindir` is that it is the randomly
      generated path for the test run, and instead of Windows Firewall remembering the
      executable signature or image hash when allowing the process to open a server
      port, it apparently remembers the image path.  That means every run will trigger
      a popup to allow it, which is bad for firing off a test run and walking away.
      
      I tried to symlink to the python executable, but that currently requires admin
      priviledges[1].  This will prompt the first time if the underlying python binary
      has never opened a server port, but appears to avoid it on subsequent runs.
      
      [1] https://bugs.python.org/issue40687
      
      Differential Revision: https://phab.mercurial-scm.org/D9815
      3e23794b9e1c
  4. Dec 17, 2020
  5. Nov 23, 2020
  6. Jul 06, 2020
  7. Nov 06, 2019
  8. Sep 24, 2018
    • Yuya Nishihara's avatar
      rust-chg: add project skeleton · aab43d5861bb
      Yuya Nishihara authored
      This directory will host the reimplementation of cHg in Rust. It will use
      Tokio [1], and tokio-hglib [2] which I wrote for an oxidized CHg, no idea
      if there's such carbon bonding in nature btw.
      
       [1]: https://tokio.rs/
       [2]: https://bitbucket.org/yuja/tokio-hglib/
      
      The reasoning for depending on Tokio is that it will allow us to handle Unix
      signals in a safer way. Well, I believed that until I found a weird function,
      handlestopsignal(), in cHg codebase. It resends the same signal to the same
      process by temporarily masking the handler, which can't be inherently async.
      So the signal handlers will stay in C, which means there isn't actually much
      reason to write async code right now, other than I've already done most of
      the async stuff, and slightly easier pager handling.
      
      The reasoning for the rewrite is that it will eventually be possible to port
      server-side config validation back to the client side, which will reduce
      the complexity of the current daemon management. It will also encourage us
      to write frontend library (e.g. command line and config parsers) in Rust.
      
      The license is GPL2+ because it's likely to include derived work from the
      cHg of C. The rust/chg crate is excluded from the root workspace as it's
      unclear how the whole rust packages should be laid out. That can be revisited
      later.
      aab43d5861bb
  9. Sep 10, 2018
    • Boris Feld's avatar
      sparse-revlog: add a test checking revlog deltas for a churning file · 4ca7a67c94c8
      Boris Feld authored
      The test repository contains 5000 revisions and is therefore slow to build:
      five minutes with CHG, over fifteen minutes without. It is too slow to build
      during the test.  Bundling all content produce a sizeable result, 20BM, too
      large to be committed. Instead, we commit a script to build the expected
      bundle and the test checks if the bundle is available. Any run of the script
      will produce the same repository content, using resulting in the same hashes.
      
      Using smaller repositories was tried, however, it misses most of the cases we
      are planning to improve. Having them in a 5000 repository is already nice, we
      usually see these case in repositories in the order of magnitude of one
      million revisions.
      
      This test will be very useful to check various changes strategy for building
      delta to store in a sparse-revlog.
      
      In this series we will focus our attention on the following metrics:
      
      The ones that will impact the final storage performance (size, space):
      * size of the revlog data file (".hg/store/data/*.d")
      * chain length info
      
      The ones that describe the deltas patterns:
      * number of snapshot revision (and their level)
      * size taken by snapshot revision (and their level)
      4ca7a67c94c8
  10. May 12, 2018
  11. Jan 10, 2018
    • Gregory Szorc's avatar
      rust: implementation of `hg` · 964212780daf
      Gregory Szorc authored
      This commit provides a mostly-working implementation of the
      `hg` script in Rust along with scaffolding to support Rust in
      the repository.
      
      If you are familiar with Rust, the contents of the added rust/
      directory should be pretty straightforward. We create an "hgcli"
      package that implements a binary application to run Mercurial.
      The output of this package is an "hg" binary.
      
      Our Rust `hg` (henceforth "rhg") essentially is a port of the existing
      `hg` Python script. The main difference is the creation of the embedded
      CPython interpreter is handled by the binary itself instead of relying
      on the shebang. In that sense, rhg is more similar to the "exe wrapper"
      we currently use on Windows. However, unlike the exe wrapper, rhg does
      not call the `hg` Python script. Instead, it uses the CPython APIs to
      import mercurial modules and call appropriate functions. The amount of
      code here is surprisingly small.
      
      It is my intent to replace the existing C-based exe wrapper with rhg.
      Preferably in the next Mercurial release. This should be achievable -
      at least for some Mercurial distributions. The future/timeline for
      rhg on other platforms is less clear. We already ship a hg.exe on
      Windows. So if we get the quirks with Rust worked out, shipping a
      Rust-based hg.exe should hopefully not be too contentious.
      
      Now onto the implementation.
      
      We're using python27-sys and the cpython crates for talking to the
      CPython API. We currently don't use too much functionality of the
      cpython crate and could have probably cut it out. However, it does
      provide a reasonable abstraction over unsafe {} CPython function
      calls. While we still have our fair share of those, at least we're
      not dealing with too much refcounting, error checking, etc. So I
      think the use of the cpython crate is justified. Plus, there is
      not-yet-implemented functionality that could benefit from cpython. I
      see our use of this crate only increasing.
      
      The cpython and python27-sys crates are not without their issues.
      The cpython crate didn't seem to account for the embedding use case
      in its design. Instead, it seems to assume that you are building
      a Python extension. It is making some questionable decisions around
      certain CPython APIs. For example, it insists that
      PyEval_ThreadsInitialized() is called and that the Python code
      likely isn't the main thread in the underlying application. It
      is also missing some functionality that is important for embedded
      use cases (such as exporting the path to the Python interpreter
      from its build script). After spending several hours trying to
      wrangle python27-sys and cpython, I gave up and forked the project
      on GitHub. Our Cargo.toml tracks this fork. I'm optimistic that
      the upstream project will accept our contributions and we can
      eventually unfork.
      
      There is a non-trivial amount of code in our custom Cargo build
      script. Our build.rs (which is called as part of building the hgcli
      crate):
      
      * Validates that the Python interpreter that was detected by the
        python27-sys crate provides a shared library (we only support
        shared library linking at this time - although this restriction
        could be loosened).
      * Validates that the Python is built with UCS-4 support. This ensures
        maximum Unicode compatibility.
      * Exports variables to the crate build allowing the built crate to e.g.
        find the path to the Python interpreter.
      
      The produced rhg should be considered alpha quality. There are several
      known deficiencies. Many of these are documented with inline TODOs.
      
      Probably the biggest limitation of rhg is that it assumes it is
      running from the ./rust/target/<target> directory of a source
      distribution. So, rhg is currently not very practical for real-world
      use. But, if you can `cargo build` it, running the binary *should*
      yield a working Mercurial CLI.
      
      In order to support using rhg with the test harness, we needed to hack
      up run-tests.py so the path to Mercurial's Python files is set properly.
      The change is extremely hacky and is only intended to be a stop-gap
      until the test harness gains first-class support for installing rhg.
      This will likely occur after we support running rhg outside the
      source directory.
      
      Despite its officially alpha quality, rhg copes extremely well with
      the test harness (at least on Linux). Using
      `run-tests.py --with-hg ../rust/target/debug/hg`, I only encounter
      the following failures:
      
      * test-run-tests.t -- Warnings emitted about using an unexpected
        Mercurial library. This is due to the hacky nature of setting the
        Python directory when run-tests.py detected rhg.
      * test-devel-warnings.t -- Expected stack trace missing frame for `hg`
        (This is expected since we no longer have an `hg` script!)
      * test-convert.t -- Test running `$PYTHON "$BINDIR"/hg`, which obviously
        assumes `hg` is a Python script.
      * test-merge-tools.t -- Same assumption about `hg` being executable with
        Python.
      * test-http-bad-server.t -- Seeing exit code 255 instead of 1 around
        line 358.
      * test-blackbox.t -- Exit code 255 instead of 1.
      * test-basic.t -- Exit code 255 instead of 1.
      
      It certainly looks like we have a bug around exit code handling. I
      don't think it is severe enough to hold up review and landing of this
      initial implementation. Perfect is the enemy of good.
      
      Differential Revision: https://phab.mercurial-scm.org/D1581
      964212780daf
  12. Nov 21, 2017
    • Gregory Szorc's avatar
      run-tests: mechanism to report exceptions during test execution · bd8875b6473c
      Gregory Szorc authored
      Sometimes when running tests you introduce a ton of exceptions.
      The most extreme example of this is running Mercurial with Python 3,
      which currently spews thousands of exceptions when running the test
      harness.
      
      This commit adds an opt-in feature to run-tests.py to aggregate
      exceptions encountered by `hg` when running tests.
      
      When --exceptions is used, the test harness enables the
      "logexceptions" extension in the test environment. This extension
      wraps the Mercurial function to handle exceptions and writes
      information about the exception to a random filename in a directory
      defined by the test harness via an environment variable. At the
      end of the test harness, these files are parsed, aggregated, and
      a list of all unique Mercurial frames triggering exceptions is
      printed in order of frequency.
      
      This feature is intended to aid Python 3 development. I've only
      really tested it on Python 3. There is no shortage of improvements
      that could be made. e.g. we could write a separate file containing
      the exception report - maybe even an HTML report. We also don't
      capture which tests demonstrate the exceptions, so there's no turnkey
      way to test whether a code change made an exception disappear.
      Perfect is the enemy of good. I think the current patch is useful
      enough to land. Whoever uses it can send patches to imprve its
      usefulness.
      
      Differential Revision: https://phab.mercurial-scm.org/D1477
      bd8875b6473c
  13. Jun 24, 2017
  14. Jun 23, 2017
  15. Aug 12, 2016
  16. Apr 25, 2017
    • Boris Feld's avatar
      packaging: add make target for linux wheels · fcddcf442e4f
      Boris Feld authored
      Having linux wheels is going to helps system without compiler or python-dev
      plus speed up the installation for everyone.
      
      I followed the manylinux example repository
      https://github.com/pypa/python-manylinux-demo
      to add a make target (build-linux-wheels) using
      official docker image to build python 2 linux wheels
      for mercurial. It generates Python 2.6 and Python 2.7 for both
      32 and 64 bits architectures.
      
      I had to blacklist several test cases for various reasons:
      * test-convert-git.t and test-subrepo-git.t because of the git version
      * test-patchbomb-tls.t because of warning using tls 1.0
        It's likely because the docker image is based on centos 5.0 and
        openssl is outdated.
      fcddcf442e4f
  17. Sep 29, 2016
    • Philippe Pepiot's avatar
      perf: add asv benchmarks · cff0f5926797
      Philippe Pepiot authored
      Airspeed velocity (ASV) is a python framework for benchmarking Python packages
      over their lifetime. The results are displayed in an interactive web frontend.
      
      Add ASV benchmarks for mercurial that use contrib/perf.py extension that could
      be run against multiple reference repositories.
      
      The benchmark suite now includes revsets from contrib/base-revsets.txt with
      variants, perftags, perfstatus, perfmanifest and perfheads.
      
      Installation requires asv>=0.2, python-hglib and virtualenv
      
      This is part of PerformanceTrackingSuitePlan
      https://www.mercurial-scm.org/wiki/PerformanceTrackingSuitePlan
      cff0f5926797
  18. Apr 27, 2016
  19. Mar 09, 2016
    • timeless's avatar
      setup: create a module for the modulepolicy · 17b85d739b62
      timeless authored
      Instead of rewriting __init__ to define the modulepolicy,
      write out a __modulepolicy__.py file like __version__.py
      
      This should work for both system-wide installation and in-place build. Therefore
      we can avoid relying on two separate modulepolicy rules, '@MODULELOADPOLICY@'
      and 'mercurial/modulepolicy'.
      17b85d739b62
  20. Feb 24, 2016
    • David R. MacIver's avatar
      testing: generate tests operations using Hypothesis · f75f7d39cca3
      David R. MacIver authored
      The idea of this patch is to expand the use of Hypothesis
      within Mercurial to use its concept of "stateful testing".
      
      The result is a test which runs a sequence of operations
      against a Mercurial repository. Each operation is given a
      set of allowed ways it can fail. Any other non-zero exit
      code is a test failure.
      
      At the end, the whole sequence is then reverified by
      generating a .t test and testing it again in pure
      mode (this is also useful for catching non-determinism
      bugs).
      
      This has proven reasonably effective at finding bugs,
      and has identified two problems in the shelve extension
      already (issue5113 and issue5112).
      f75f7d39cca3
  21. Jan 03, 2016
  22. Jan 05, 2016
  23. Oct 13, 2014
  24. Jul 05, 2014
  25. May 27, 2014
  26. Nov 05, 2013
  27. Jul 03, 2013
  28. Aug 07, 2012
    • Adrian Buehlmann's avatar
      exewrapper: adapt for legacy HackableMercurial · 93d97a212559
      Adrian Buehlmann authored
      We give up using CPython's PythonXX.lib import libraries (and Python.h), and
      now "manually" call the LoadLibrary() / GetProcAddress() Windows API's instead.
      
      If there is a "hg-python" subdirectory (the canonical directory name for
      HackableMercurial's private Python copy) next to the hg.exe, we load the
      pythonXX.dll from there (feeding an absolute path to LoadLibrary) and we set
      Py_SetPythonHome() to that directory, so that the Python libraries are used
      from there as well.
      
      If there is no "hg-python" subdir found next to the hg.exe, we do not feed an
      absolute path to LoadLibrary. This continues to allow to find a globally
      installed Python DLL, as before this change - that is, without having to edit,
      delete, rename, or configure anything.
      
      Note that the hg.exe built is still bound to a *specific* major version of the
      pythonXX.dll (e.g. python27.dll). What version it is, is inferred from the
      version of the python interpreter that was used when calling setup.py. For
      example
      
        C:\python27_x86\python.exe setup.py build_hgexe -i --compiler=mingw32
      
      builds a hg.exe (using the mingw32 tool chain) bound to (x86) Python 2.7. And
      
        C:\python27_x86\python.exe setup.py build_hgexe -i
      
      builds the same using the Microsoft C compiler/linker. (Note that the Microsoft
      toolchain combined with x64 CPython can be used to build an x64 hg.exe.)
      
      setup.py is changed to write the name of the pythonlib into the generated header
      file "mercurial/hgpythonlib.h", which is #included by exewrapper.c. For a Python
      2.7 build, it for example contains:
      
        #define HGPYTHONLIB "python27"
      
      exewrapper.c then uses HGPYTHONLIB for the name of the Python dll to load.
      
      We don't want to track mercurial/hgpythonlib.h, so we add it to .hgignore.
      93d97a212559
  29. Jul 29, 2012
  30. Jul 01, 2012
  31. Apr 27, 2012
  32. Jan 11, 2012
  33. Dec 20, 2011
  34. Jun 10, 2011
  35. Jun 04, 2011
  36. Mar 14, 2011
  37. Feb 03, 2011
  38. Jun 18, 2010
  39. Apr 24, 2009
    • Bryan O'Sullivan's avatar
      win32text: be more careful about rejecting violating changesets · 441dc7becd43
      Bryan O'Sullivan authored
      We now try to walk changesets in reverse order from newest to oldest,
      so that if we see a file multiple times, we treat the newest version
      as canonical.
      
      This should prevent us from rejecting a changegroup that contains an
      unacceptable commit followed later by a commit that fixes the problem.
      441dc7becd43
Loading