target repo have about 43000 revision. and it imports with a speed about 4000revisions/hour. python got 30% cpu on i5.
after 10hours, speed degrade for 1000revs/8hours
it`s terrible
Edited
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Child items
0
Show closed items
No child items are currently assigned. Use child items to break down this issue into smaller parts.
Yeah, I think hg-git has some spurious slowdowns somewhere I just merged !44 (merged) which ensures that it always saves commit map after a pull. With this, you can work around it — or check if the slowdowns are “genuine” — by creating an empty repo and setting the path. Then, you do a pull, which you can safely interrupt and restart.
(I also merged !47 (merged) which might be the cause of the slowdown.)
If the clone is still going, please note that interrupting it will delete the intermediate results. You may be able to avoid that by copying the entire repository prior to interrupting it; however, if no git-mapfile exists within .hg, everything will still get pulled anew. Sorry.
i've tryed import merge-requests from here, but looks that i miss extention topic, and can't found it anywhere. can you explain - how to pull branches that assigned to MR?
Well, you shouldn't need it, I merged everything into default (topics are part of the evolve project, but only necessary if you want to contribute to a project on Heptapod.)
(writing this while cloning the example to compare the latest hg-git with 0.9 on this).
hg-git is also quite slow in the other direction (exporting to Git), perhaps a bit better than what's reported here.
Out of memory, on reasonably fast hardware, the rate is at about 5-10 commits per second if I recall correctly. As an example, the various import attempts I made for PyPy (about 100k changesets) in Heptapod (that involves conversion to Git) took between 4 and 6 hours to complete. There is indeed a slowdown after a while, ending at 2-3 per second. Of course, it depends on actual changeset content and probably lots of different factors.
Here are my measurements, cloning the mbed-os repo (not by any means scientific)
I tried with hg-git 0.9.0 and 8de310e8ec98 (current head of the default branch).
Both took roughly one hour on my workstation, including network. The difference is around 2 minutes, which has to be considered noise under the circumstances.
It took 5 minutes to import the last 2000 commits with 0.9.0. At this rate the whole import would then have taken a bit over 100 minutes instead of one hour, so there is a slow down, but it wasn't that bad in my case.
This is a fairly fast machine (AMD Ryzen 5 1600) with 32GB of RAM.
I'm guessing @gracinet is using Linux; I'm using a Mac. Do you have other extensions? What is the output of hg show extensions? What is the output of hg debuginstall?
EDIT:hg show git hggit might also be relevant; particularly if you've set git.findcopiesharder which can cause a rather massive slowdown.
I ran these in a virtualenv that I created for the occasion, with hg 5.6:
Here's a rerun I did for RAM afterwards. It took up to 6GB
(testenv3) ~/octobus/hg-git/hg-git $ pip freezecertifi==2020.11.8dulwich==0.20.11-e hg+ssh://foss.heptapod.net/mercurial/hg-git@8de310e8ec98c8f5cf422c09e1a9013cbc10b297#egg=hg_gitmercurial==5.6urllib3==1.26.2$ date; /usr/bin/time --verbose /tmp/testenv3/bin/hg clone git+ssh://git@github.com:ARMmbed/mbed-os.git --config extensions.hggit= mbed-os-hg-git-8de310e8ec98c8f5cf422c09e1a9013cbc10b297Sat 21 Nov 2020 06:35:54 PM CET*** failed to import extension evolve: No module named 'evolve'*** failed to import extension topic: No module named 'topic'importing git objects into hgupdating to branch default 16793 files updated, 0 files merged, 0 files removed, 0 files unresolved Command being timed: "/tmp/testenv3/bin/hg clone git+ssh://git@github.com:ARMmbed/mbed-os.git --config extensions.hggit= mbed-os-hg-git-8de310e8ec98c8f5cf422c09e1a9013cbc10b297" User time (seconds): 3727.25 System time (seconds): 146.27 Percent of CPU this job got: 84% Elapsed (wall clock) time (h:mm:ss or m:ss): 1:16:07 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 6301432 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 11 Minor (reclaiming a frame) page faults: 3199289 Voluntary context switches: 210412 Involuntary context switches: 1575276 Swaps: 0 File system inputs: 1760 File system outputs: 34527000 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0
I've installed more stuff in that virtualenv since then (because I happen to create lots temporary such virtualenvs).
(venv.py38) D:\projects\hg\thg\thg.work>hg versionextension 'split' overrides commands: splitextension 'uncommit' overrides commands: uncommitMercurial Distributed SCM (version 5.6.1)(see https://mercurial-scm.org for more information)Copyright (C) 2005-2020 Matt Mackall and othersThis is free software; see the source for copying conditions. There is NOwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.(venv.py38) D:\projects\hg\thg\thg.work>
(venv.py38) D:\projects\hg\hg-git>pip freezeWARNING: Could not generate requirement for distribution -ip 20.1.1 (d:\projects\hg\thg\thg.work\venv.py38\lib\site-packages): Parse error at "'-ip==20.'": Expected W:(abcd...)certifi==2019.9.11comtypes==1.1.7dulwich==0.20.5hg-evolve==10.0.0hg-fluiddyn==0.0.3hg-git==0.9.0importlib-metadata==1.6.0iniparse==0.5keyring==21.2.1mercurial==5.6.1mercurial-extension-utils==1.5.0mercurial-keyring==1.3.0py2exe==0.9.2.2PyQt5==5.13.2PyQt5-sip==12.7.0pywin32==227pywin32-ctypes==0.2.0QScintilla==2.10.8six==1.15.0urllib3==1.25.9zipp==3.1.0
hg show extensions and hg show git hggit - show nothing.
behaviour of my gh-git still same: it goes slower and slower, till moment when go a few revision/sec.
To elaborate a bit: Current indications are that the default branch fixes this, although it won't work with the currently released TortoiseHg due to mercurial/tortoisehg/thg#5766 (closed).