- Apr 22, 2021
-
-
Georges Racinet authored
-
- Apr 11, 2021
-
-
Sushil Khanchi authored
-
Sushil Khanchi authored
There was a case, found while writing gitaly comparison test for FindCommits (next patch) where current DAG logic missed some cset to include. r1...r2 revspec means, csets ancestor of r1 or r2, but not ancestor of both. Graph: D | \ B C | | | / A | O revspec: 'D...B' Expected revs: (D, C) Logic before this patch would return: (D,)
-
Sushil Khanchi authored
-
Sushil Khanchi authored
The change around offset make sure that we follow along with Gitaly and give priority to offset first, then limit. If no limit is passed, then request.limit is Zero.
-
Sushil Khanchi authored
-
- Apr 22, 2021
-
-
Georges Racinet authored
It seems that the current stable branch of hg-evolve is not ready for the current stable branch of Mercurial (5.8rc)
-
- Apr 15, 2021
-
-
Georges Racinet authored
-
Georges Racinet authored
-
Georges Racinet authored
-
Georges Racinet authored
The forking that the `hgitaly-serve` command is doing by default is harmful when we want to debug with pdb: request handling always end in an error. The new `--mono-process` option solves that by serving directly from the main process. This is probably a matter of forwarding all the right file descriptors for the tty, but it's not worth the investigation time when there is such a simple solution. Could have done it in the command itself, but this is probably easier to maintain in the `server` module and certainly easier to test. No strong assertions that it doesn't fork in the test, but coverage proves that the early return got executed; that should be enough.
-
Georges Racinet authored
Previous fixes have been developed and measured in a Python 3.9 virtualenv (HDK). Replaying the measurements in a Python 3.8 HDK (the only supported Python version in all current Heptapod branches) immediately showed that this wasn't enough. After reconstruction of the virtualenv with Python 3.8, it became obvious that repository instances (and classes) where doubled, and the reason was the systematic instantiation of a sharedrepo source. Applying the same treatment for the latter got us back to the improvement previously obtained. It seems frankly dubious that the Python version is really implicated, but it could happen: we're after all dealing with stuff that the GC didn't clear by itself. Still, it could simply have been a matter of library version (not Mercurial, it's the same one).
-
Georges Racinet authored
For something similar with hgweb, look at changeset ff2370a70fe8 in Mercurial. Surprisingly, this does not change the memory footprint in our current investigations (which is just using examples/client.py with more calls). This means that the regular garbage collector was able to reap benefits of the parent changeset (calling `gc.collect()` is supposed to be more aggressive on the oldest generation). Still, keeping this as a safety net because our example certainly wouldn't cover all possible situations With rates of 100 and 500, the new log lines show us the GC working, with more to collect at 500 than at 100. But at 1000, we get back to the same amounts of collected objects and time spent than with 100. This suggests that this is close to the natural rate, hence we aren't really harming performance. In any case, this is a constant, and we could make it more configurable if needed.
-
Georges Racinet authored
This brings down the leaking over 1000 requests from 55MB to about 5MB. We are not yet forcing garbage collection (as `hgwebdir` does). The `_filteredrepotypes` keeps a correspondence between unfiltered repository classes and the filtered version. The problem is that each new repository is an instance of the single-usage class generated in `hgext3rd.topic.reposetup`, so these accumulate in `_filteredrepotypes`. Ironically the mapping seems to be there to avoid some other case of leaking: ``` # Python <3.4 easily leaks types via __mro__. See # https://bugs.python.org/issue17950. We cache dynamically created types # so they won't be leaked on every invocation of repo.filtered(). _filteredrepotypes = weakref.WeakKeyDictionary() ``` One could imagine that garbage collection of the repo would also just clear its reference from the `WeakKeyDictionary`, but it doesn't happen. A final suprise is that this unlocks the collection of `ui` instances. According to investigation made with `guppy3` the class objects were actually the ones eating the most memory, but getting rid of `ui` instances is also good news.
-
Georges Racinet authored
Found out while investigating heptapod#466 that actually the machinery behind `hg.repository` actually performs a copy on its own. Our copy was adding pressure on the Python memory system. This does not cure the memory leak, but it saves about 2MB of leak over 1000 requests, even though it doesn't change the fact that none of these ui instances are actually collected (seen through instrumentation not part of this changeset). From `localrepo.py` in Mercurial 5.6.1: ``` def makelocalrepository(baseui, path, intents=None): """Long docstring not cited here.""" ui = baseui.copy() # Prevent copying repo configuration. ui.copy = baseui.copy (...) ``` Note that this last line actually prevents `baseui` ( would be the copy we were creating) to be freed immediately. makelocalrepository in turn is called via this backtrace: ``` venv/lib64/python3.9/site-packages/mercurial/hg.py(221)repository() -> peer = _peerorrepo( venv/lib64/python3.9/site-packages/mercurial/hg.py(188)_peerorrepo() -> obj = _peerlookup(path).instance( venv/lib64/python3.9/site-packages/mercurial/localrepo.py(3214)instance() -> return makelocalrepository(ui, localpath, intents=intents) > venv/lib64/python3.9/site-packages/mercurial/localrepo.py(517)makelocalrepository() -> ui = baseui.copy() ```
-
Georges Racinet authored
This will make instrumentation for memory investigation easier.
-
- Apr 01, 2021
-
-
Georges Racinet authored
-
Georges Racinet authored
-
Georges Racinet authored
-
- Mar 14, 2021
-
-
Sushil Khanchi authored
-
Sushil Khanchi authored
-
Sushil Khanchi authored
-
Sushil Khanchi authored
-
- Mar 19, 2021
-
-
Georges Racinet authored
Since we're about to add at least one new service method, next version on the stable branch will be 0.11
-
Georges Racinet authored
-
- Mar 12, 2021
-
-
Georges Racinet authored
-
Georges Racinet authored
Found out about this while testing with the Rails app. Of course, it is natural for symbolic refs, and doesn't sound very logical with normal refs, a result of under-specification ultimately due to the fact that Gitaly simply uses `git update-ref`. We keep refusing ref target revisions for keep-arounds because we are much surer that the Rails app will only use full SHAs as target revisions. Also, it swallows exceptions, avoiding failures if `WriteRef` refuses to create the keep-around (see `Gitlab::Git::KeepAround`). We should have a Gitaly comparison test for this, but we already have RSpec tests in the Rails app, so this can be in a follow-up.
-
Georges Racinet authored
The work to support special refs is well worth a minor version update.
-
- Mar 11, 2021
-
-
Georges Racinet authored
Usually, I would stand against wide exception catching, even with proper logging. But in this case, we're about (Heptapod 0.20.2) to have the Rails app repeat all special refs creations to HGitaly, and it should soon (Heptapod 0.21) even be the case for non-native Mercurial projects. Currently (HGitaly 0.9), all special refs creation do fail for native Mercurial projects, and that is no problem, with a major exception: Merge Requests refs are created by an actual Git fetch and are really needed.
-
Georges Racinet authored
reminder: all current projects handled with HGitaly have a Git repo.
-
Georges Racinet authored
Closes #52
-
Georges Racinet authored
Without an explicit return, it was giving a gRPC error
-
- Mar 09, 2021
-
-
Georges Racinet authored
-
- Mar 10, 2021
-
-
Georges Racinet authored
Closes #50
-
- Mar 03, 2021
-
-
Georges Racinet authored
This will support direct special refs creation from the Rails app, but not the indirect ones, such as created in the Git case by `FetchSourceBranch`. Keep-arounds are taken into account. Also we still don't enforce the existence of the special refs state file yet (see discussion about transitional aspects in heptapod#431). Closes #48
-
Georges Racinet authored
except keep-arounds for which it's unlikely to be useful (py-heptapod only allows to list them)
-
Georges Racinet authored
See heptapod#431 for more explanations about what we call special refs, and the problem to solve. This depends on primitives in py-heptapod that are currently unreleased, whence the version bump. We don't enforce the existence of the special refs state file yet.
-
- Mar 10, 2021
-
-
Georges Racinet authored
-
- Mar 03, 2021
-
-
Georges Racinet authored
We'll need both the `str` and the `bytes` case. For balance of pleasure and pain, SHA-2 is accepted.
-
- Mar 11, 2021
-
-
Georges Racinet authored
Lately these jobs haven't been running for MR pipelines. Let's try and force it by explicit rules. Also, have check-sdist run only for MR pipelines and pushes to the main development branches.
-