hg-git issueshttps://foss.heptapod.net/mercurial/hg-git/-/issues2024-03-21T19:02:13Zhttps://foss.heptapod.net/mercurial/hg-git/-/issues/428Initialising a git repo from a Mercurial repo which includes a bookmark 'main...2024-03-21T19:02:13ZNorman GrayInitialising a git repo from a Mercurial repo which includes a bookmark 'main' produces git warning messagesI tried creating a git repo from a Mercurial repo which (semi-inadvertently) had a bookmark `main` (you can probably see where this is going...).
The initialisation worked in technical terms (it appears), but produces scary warning mess...I tried creating a git repo from a Mercurial repo which (semi-inadvertently) had a bookmark `main` (you can probably see where this is going...).
The initialisation worked in technical terms (it appears), but produces scary warning messages. If I try the same with a Mercurial tag `master`, then this is differently odd, and seems to leave the target git repository in a slightly unexpected state. This state is probably recoverable in git terms, but is not what a user might be expecting (especially if they treasure their git ignorance, or are cut-and-pasting instructions from the web (I know...)).
This is all somewhat intelligible in retrospect, but it took some headscratching to work out what might be going wrong, and required me knowing that `main` and `master` are magic branch names for git, unlike for Mercurial. It might be worth displaying a prominent user warning if `hg-git` finds itself asked to do this.
For example:
```
% hg log
changeset: 0:27e1910f7815
bookmark: hg
bookmark: main
tag: tip
user: Me <me@example.org>
date: Thu Mar 21 18:31:23 2024 +0000
summary: First
% hg bookmark
* hg 0:27e1910f7815
main 0:27e1910f7815
% (mkdir ../git-repo; cd ../git-repo; git init)
Initialized empty Git repository in /private/tmp/git-repo/.git/
% hg push ../git-repo
pushing to ../git-repo
searching for changes
adding objects
remote: found 0 deltas to reuse
remote: error: refusing to update checked out branch: refs/heads/main
remote: error: By default, updating the current branch in a non-bare repository
remote: is denied, because it will make the index and work tree inconsistent
remote: with what you pushed, and will require 'git reset --hard' to match
remote: the work tree to HEAD.
remote:
remote: You can set the 'receive.denyCurrentBranch' configuration variable
remote: to 'ignore' or 'warn' in the remote repository to allow pushing into
remote: its current branch; however, this is not recommended unless you
remote: arranged to update its work tree to match what you pushed in some
remote: other way.
remote:
remote: To squelch this message and still keep the default behaviour, set
remote: 'receive.denyCurrentBranch' configuration variable to 'refuse'.
added 1 commits with 1 trees and 1 blobs
adding reference refs/heads/hg
warning: failed to update refs/heads/main; branch is currently checked out
% cd ../git-repo
% git status
On branch main
No commits yet
nothing to commit (create/copy files and use "git add" to track)
% git log
fatal: your current branch 'main' does not have any commits yet
% git checkout -b main hg
Already on 'main'
```
Similarly, with a Mercurial tag `master`:
```
% hg log
changeset: 0:27e1910f7815
bookmark: hg
bookmark: master
tag: tip
user: Me <me@example.org>
date: Thu Mar 21 18:31:23 2024 +0000
summary: First
% hg bookmark
* hg 0:27e1910f7815
master 0:27e1910f7815
% (mkdir ../git-repo; cd ../git-repo; git init)
Initialized empty Git repository in /private/tmp/git-repo/.git/
% hg push ../git-repo
pushing to ../git-repo
searching for changes
adding objects
remote: found 0 deltas to reuse
added 1 commits with 1 trees and 1 blobs
adding reference refs/heads/hg
adding reference refs/heads/master
% cd ../git-repo
% git status
On branch main
No commits yet
nothing to commit (create/copy files and use "git add" to track)
% git log
fatal: your current branch 'main' does not have any commits yet
% git checkout -b master hg
fatal: a branch named 'master' already exists
```https://foss.heptapod.net/mercurial/hg-git/-/issues/427`git.authors` config setting is not always used2024-02-24T21:45:55ZMingun`git.authors` config setting is not always usedI tried to convert my mercurial repo to git and I want to change my name (`Mingun`) that set in mercurial commits to my git name (`Mingun <my@e.mail>`). To to that I've created the `git-authors.txt` file in the root of my mercurial repo:...I tried to convert my mercurial repo to git and I want to change my name (`Mingun`) that set in mercurial commits to my git name (`Mingun <my@e.mail>`). To to that I've created the `git-authors.txt` file in the root of my mercurial repo:
```
Mingun = Mingun <my@e.mail>
```
Then I edited `<repo>/.hg/hgrc` file:
```
[paths]
default = https://bitbucket.org/Mingun/peg-grammars
[git]
authors = git-authors.txt
```
Then I run the following commands:
```bash
$ cd <repo>
<repo> $ hg bookmarks hg
<repo> $ git init --bare ../grammars-bare-git
Initialized empty Git repository in D:/Projects/grammars-bare-git/
<repo> $ hg push --verbose ../grammars-bare-git
pushing to ../grammars-bare-git
use authors mapping file D:\Projects\grammars-hg/git-authors.txt # My addition
author mapping: "Mingun" -> "Mingun <my@e.mail>" # My addition
finding unexported changesets
total: 0 # My addition
searching for changes
remote: counting objects: 2153, done.
539 commits found
adding objects
remote: counting objects: 2153, done.
remote: found 0 deltas to reuse
added 539 commits with 820 trees and 794 blobs
adding reference ::refs/heads/hg => GIT:ef274397
adding reference ::refs/tags/ParserSupportImport => GIT:b03e4e1e
<repo> $ git clone -b hg ../grammars-bare-git ../grammars-git
```
After checking the commit info in git I noted that author is set to `Mingun <none@none>`. I've debug a bit and found that [`export_hg_commit`](https://foss.heptapod.net/mercurial/hg-git/-/blob/cf50e7bed8a181c010f9153341f838aadd707415/hggit/git_handler.py#L711) which does the mapping is not called in my case because [`total`](https://foss.heptapod.net/mercurial/hg-git/-/blob/cf50e7bed8a181c010f9153341f838aadd707415/hggit/git_handler.py#L673) variable is `0`. In the log above you can see my debug output which is added by this patch:
```diff
diff -r cf50e7bed8a1 hggit/git_handler.py
--- a/hggit/git_handler.py Sat Jan 13 10:59:28 2024 +0100
+++ b/hggit/git_handler.py Sun Feb 25 00:16:40 2024 +0500
@@ -273,13 +273,18 @@
self.author_map = {}
authors_path = self.ui.config(b'git', b'authors')
if authors_path:
- with open(self.repo.wvfs.join(authors_path), 'rb') as f:
+ path = self.repo.wvfs.join(authors_path)
+ with open(path, 'rb') as f:
+ self.ui.note(_(b'use authors mapping file %s\n') % path)
for line in f:
line = line.strip()
if not line or line.startswith(b'#'):
continue
from_, to = RE_AUTHOR_FILE.split(line, 2)
+ self.ui.note(_(b'author mapping: "%s" -> "%s"\n') % (from_, to))
self.author_map[from_] = to
+ else:
+ self.ui.note(_(b'authors mapping file is not specified\n'))
# FILE LOAD AND SAVE METHODS
@@ -671,6 +676,7 @@
export.append(ctx)
total = len(export)
+ self.ui.note(_(b' total: %d\n') % total)
if not total:
return
@@ -729,6 +735,7 @@
commit.author_time = int(time)
commit.author_timezone = -timezone
+ self.ui.note(_(b" git author will be: %s\n") % commit.author)
if b'committer' in extra:
try:
# fixup timezone
```https://foss.heptapod.net/mercurial/hg-git/-/issues/426local clone out of sync with github repo2024-01-24T03:38:29ZErnie Raellocal clone out of sync with github repo$ hg --version -v
Mercurial Distributed SCM (version 6.6.2)
hggit external 1.1.0 (dulwich 0.21.7)
Given two repos on github: `gh-main`, `gh-fork`;
and two local clones of them: `local-main`, `local-fork`.
On `local-fork`:
``...$ hg --version -v
Mercurial Distributed SCM (version 6.6.2)
hggit external 1.1.0 (dulwich 0.21.7)
Given two repos on github: `gh-main`, `gh-fork`;
and two local clones of them: `local-main`, `local-fork`.
On `local-fork`:
```
$ hg in
comparing with git@github.com:errael/swingset.git
no changes found
$ hg pull
pulling from git@github.com:errael/swingset.git
no changes found
$ hg glog
...
| x changeset: 1287:38d227b9764b
|/ bookmark: SSV
| tag: default/SSV
| obsolete: pruned using prune
| summary: Add SSVersion.
```
NOTE: the above `local-fork` changeset `1287` has the **same git hash as on github**.
Is there a way to revive `1287`? `hg touch` says you get a different `changeset ID`.
My understanding is that a different `ID` gets the local clone out of sync with the github repo.
_The following seems to be a bug, since **garbage is printed**, other changesets do it, so seems unrelated
to this problem_.
```
$ hg pull -B SSV
pulling from git@github.com:errael/swingset.git
abort: unknown revision '8�'�vK+���~�Gɓ��('
```
_Back to the problem_.
I did a bad prune, following is how I got into this (IIRC).
There's a bookmark SSV, that originated in local-main and was pushed to gh-main.
Sitting in `local-fork`, did `hg pull -r SSV local-main`
and ended up with bookmarks
```
SSV@local-main
SSV
```
Oops, not the goal.
At this point, maybe just `hg pull -B SSV local-main` would have fixed everything.
But instead I pruned `SSV@local-main`, then did the bookmark pull,
and then "hg push -f -B SSV", which pushed to gh-fork. And then both repos
on github were in sync; same `commit hash` for both `gh-main` and `gh-fork`.
So, while in `local-fork` somehow pull from `local-main` push to `gh-fork`
got github in sync. But left `local-fork` broken
**Is there a way to fix `local-fork`?**
I'll probably create a new clone from `gh-fork`, then transplant the work I've
got in `hg-fork` to it (`transplant`/`graft` will be the same in this case).
Or maybe I can just pull from the broken clone.https://foss.heptapod.net/mercurial/hg-git/-/issues/424Can't delete accidental commit on remote git repository2023-10-15T22:34:57ZErnie RaelCan't delete accidental commit on remote git repository(This might be considered a feature request, but dealing with it sure feels like a bug)
Following is an exact copy of a post made to the forum. Since then I've found that every time I sync my fork to the main repository (all of this on ...(This might be considered a feature request, but dealing with it sure feels like a bug)
Following is an exact copy of a post made to the forum. Since then I've found that every time I sync my fork to the main repository (all of this on github) and then pull to my local repo. I get new changesets which, along with several others, I want to delete from my github repository. They look something like
```
| * changeset: 18900:b32da47c387e
|/| tag: default/master
...
| | instability: orphan
| | summary: Merge branch 'vim:master' into master
```
I don't know git (and I very much thank and appreciate hg-git for that) so I'm hoping at least for some hint on what/how to do what I need.
---
I did a push to a github repo (a fork) and left out the "-B"; other things were done. I'd like to remove the changesets/commits, 18835 and 18829 (see below), from the github repo. I tried pruning locally, then push the pruned revsets, but no luck. Can't find a way to remove them using the github website.
Is there a way to clean up the github repo using hg-git? If not, any suggestions on something to look at to easily get this done? Can I use "<mylocalrepo>/.hg/git"?
-ernie
```
o changeset: 18841:76a07b084276
| bookmark: LOCAL_DEBUG_MAKEFILE
| tag: tip
| user: Ernie Rael <errael@raelity.com>
| date: Mon Aug 21 07:16:59 2023 -0700
| summary: LOCAL_DEBUG_MAKEFILE
|
o changeset: 18840:adeb4400a922
| bookmark: MsgVarDefiningClass
| parent: 18834:39523d22b15d
| user: Ernie Rael <errael@raelity.com>
| date: Tue Oct 03 19:38:44 2023 -0700
| summary: Use defining class in variable access error messages.
|
| o changeset: 18837:0e20fc19cb17
|/ bookmark: ForeachExpr1Expr2
| parent: 18834:39523d22b15d
| user: Ernie Rael <errael@raelity.com>
| date: Wed Mar 15 20:14:10 2023 -0700
| summary: Implement foreach({expr1}, {expr2})
|
| x changeset: 18835:0d50de48e9b0
|/| tag: default/master
| | parent: 18829:acb36821c198
| | parent: 18834:39523d22b15d
| | user: errael <errael@raelity.com>
| | date: Thu Oct 05 08:59:56 2023 -0700
| | obsolete: pruned
| | summary: Merge branch 'vim:master' into master
| |
@ | changeset: 18834:39523d22b15d
| | bookmark: real-master
| | user: Christian Brabandt <cb@256bit.org>
| | date: Wed Oct 04 21:58:24 2023 +0200
| | summary: patch 9.0.1984: CI: Test_open_delay*() fails on FreeBSD 14
| |
o | changeset: 18833:56947cb3f4f1
| | user: zeertzjq <zeertzjq@outlook.com>
| | date: Wed Oct 04 21:45:33 2023 +0200
| | summary: patch 9.0.1983: scrolling inactive window not possible with cursorbind
| |
o | changeset: 18832:1d19383b843c
| | parent: 18819:0c8f51a5af44
| | user: yasuda <yasuda@kyoto-sr.co.jp>
| | date: Wed Oct 04 20:50:35 2023 +0200
| | summary: runtime(netrw): error when trying to :bd unloaded buffer
| |
+---x changeset: 18830:3cfde4bbc21e
| | tag: default/MsgVarDefiningClass
| | parent: 18819:0c8f51a5af44
| | user: Ernie Rael <errael@raelity.com>
| | date: Tue Oct 03 19:38:44 2023 -0700
| | obsolete: rewritten using amend, rebase as 18840:adeb4400a922
| | summary: Use defining class in variable access error messages.
| |
| x changeset: 18829:acb36821c198
|/ bookmark: TwoMessages
| tag: default/TwoMessages
| parent: 18819:0c8f51a5af44
| user: Ernie Rael <errael@raelity.com>
| date: Wed Oct 04 19:27:01 2023 -0700
| obsolete: pruned using prune
| summary: Avoid two messages for same error.
|
o changeset: 18819:0c8f51a5af44
| user: Ernie Rael <errael@raelity.com>
| date: Wed Oct 04 20:16:22 2023 +0200
| summary: patch 9.0.1982: vim9: clean up from v9.0.1955
```https://foss.heptapod.net/mercurial/hg-git/-/issues/410AttributeError: 'changelog' object has no attribute 'nullid'2023-06-24T04:23:33ZMatt HarbisonAttributeError: 'changelog' object has no attribute 'nullid'I tried `hg serve` on a machine with the latest Prometheus clone, and tried pulling it into the repo mentioned in the previous issue, and got this with hg 6.4.2 and hg-git 1.0.2:
```
$ /c/Program\ Files/TortoiseHg/hg.exe in -r tip http:...I tried `hg serve` on a machine with the latest Prometheus clone, and tried pulling it into the repo mentioned in the previous issue, and got this with hg 6.4.2 and hg-git 1.0.2:
```
$ /c/Program\ Files/TortoiseHg/hg.exe in -r tip http://10.30.1.17:8000
comparing with http://10.30.1.17:8000/
searching for changes
** Unknown exception encountered with possibly-broken third-party extension "mercurial_keyring" unknown
** which supports versions unknown of Mercurial.
** Please disable "mercurial_keyring" and try your action again.
** If that fixes the bug please report it to https://foss.heptapod.net/mercurial/mercurial_keyring/issues
** Python 3.9.13 (tags/v3.9.13:6de2ca5, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)]
** Mercurial Distributed SCM (version 6.4.2)
** Extensions loaded: absorb, blackbox, evolve 11.0.1, extdiff, fastannotate, fix, git (pygit2 1.9.1), hggit 1.0.2 (dulwich 0.21.3), mercurial_keyring unknown, phabblocker 20230202, phabricator, rebase, show,
rip, topic 1.0.1
Traceback (most recent call last):
File "hg", line 58, in <module>
File "mercurial\dispatch.pyc", line 143, in run
File "mercurial\dispatch.pyc", line 232, in dispatch
File "mercurial\dispatch.pyc", line 276, in _rundispatch
File "mercurial\dispatch.pyc", line 451, in _runcatch
File "mercurial\dispatch.pyc", line 461, in _callcatch
File "mercurial\scmutil.pyc", line 153, in callcatch
File "mercurial\dispatch.pyc", line 441, in _runcatchfunc
File "mercurial\dispatch.pyc", line 1266, in _dispatch
File "mercurial\dispatch.pyc", line 899, in runcommand
File "mercurial\dispatch.pyc", line 1278, in _runcommand
File "mercurial\dispatch.pyc", line 1264, in <lambda>
File "mercurial\util.pyc", line 1881, in check
File "mercurial\commands.pyc", line 4418, in incoming
File "mercurial\hg.pyc", line 1410, in incoming
File "mercurial\hg.pyc", line 1365, in _incoming
File "hggit\gitrepo.pyc", line 179, in getremotechanges
File "mercurial\bundlerepo.pyc", line 614, in getremotechanges
File "mercurial\discovery.pyc", line 61, in findcommonincoming
File "mercurial\setdiscovery.pyc", line 412, in findcommonheads
AttributeError: 'changelog' object has no attribute 'nullid'
```
(I'm not sure why it's expected that `changelog` objects have a `nullid` instead of just looking at `nodemod`, but maybe this is part of the transition off SHA-1?)https://foss.heptapod.net/mercurial/hg-git/-/issues/409AttributeError: 'gitdirstate' object has no attribute 'refresh' during pull2023-09-02T05:42:41ZMatt HarbisonAttributeError: 'gitdirstate' object has no attribute 'refresh' during pullI'm trying to pull `git+https://github.com/prometheus/prometheus.git`, and got this with hg 6.4.2 + hg-git 1.0.2 (dulwich 0.21.3):
```
pulling from git+https://github.com/prometheus/prometheus.git
Traceback (most recent call last):
Fi...I'm trying to pull `git+https://github.com/prometheus/prometheus.git`, and got this with hg 6.4.2 + hg-git 1.0.2 (dulwich 0.21.3):
```
pulling from git+https://github.com/prometheus/prometheus.git
Traceback (most recent call last):
File "hgext3rd\evolve\__init__.pyc", line 563, in wrapmayobsoletewc
File "mercurial\util.pyc", line 1881, in check
File "mercurial\commands.pyc", line 5504, in pull
File "hggit\util.pyc", line 99, in inner
File "hggit\gitrepo.pyc", line 196, in exchangepull
File "hggit\git_handler.pyc", line 362, in fetch
File "mercurial\repoview.pyc", line 457, in __getattr__
File "mercurial\pycompat.pyc", line 358, in w
File "mercurial\localrepo.pyc", line 125, in __get__
File "mercurial\scmutil.pyc", line 1709, in __get__
File "mercurial\localrepo.pyc", line 1768, in changelog
File "mercurial\localrepo.pyc", line 223, in __get__
File "mercurial\util.pyc", line 1760, in __get__
File "mercurial\localrepo.pyc", line 1783, in dirstate
AttributeError: 'gitdirstate' object has no attribute 'refresh'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "mercurial\dispatch.pyc", line 461, in _callcatch
File "mercurial\scmutil.pyc", line 153, in callcatch
File "mercurial\dispatch.pyc", line 441, in _runcatchfunc
File "mercurial\dispatch.pyc", line 1266, in _dispatch
File "mercurial\dispatch.pyc", line 899, in runcommand
File "mercurial\dispatch.pyc", line 1278, in _runcommand
File "mercurial\dispatch.pyc", line 1264, in <lambda>
File "mercurial\util.pyc", line 1881, in check
File "mercurial\util.pyc", line 1881, in check
File "hgext\rebase.pyc", line 2182, in pullrebase
File "mercurial\util.pyc", line 1881, in check
File "mercurial\util.pyc", line 1881, in check
File "hgext3rd\evolve\__init__.pyc", line 565, in wrapmayobsoletewc
File "mercurial\lock.pyc", line 393, in release
File "mercurial\lock.pyc", line 376, in release
File "mercurial\localrepo.pyc", line 3125, in unlock
File "mercurial\repoview.pyc", line 457, in __getattr__
File "mercurial\pycompat.pyc", line 358, in w
File "mercurial\localrepo.pyc", line 223, in __get__
File "mercurial\util.pyc", line 1760, in __get__
File "mercurial\localrepo.pyc", line 1783, in dirstate
AttributeError: 'gitdirstate' object has no attribute 'refresh'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "mercurial\dispatch.pyc", line 276, in _rundispatch
File "mercurial\dispatch.pyc", line 451, in _runcatch
File "mercurial\dispatch.pyc", line 505, in _callcatch
File "mercurial\dispatch.pyc", line 1376, in handlecommandexception
File "mercurial\ui.pyc", line 2172, in log
File "mercurial\loggingutil.pyc", line 142, in log
File "hgext\blackbox.pyc", line 130, in log
File "hgext\blackbox.pyc", line 147, in _log
File "mercurial\localrepo.pyc", line 1947, in __getitem__
File "mercurial\extensions.pyc", line 532, in closure
File "hgext3rd\topic\__init__.pyc", line 812, in wrapinit
File "mercurial\context.pyc", line 1562, in __init__
File "mercurial\localrepo.pyc", line 223, in __get__
File "mercurial\util.pyc", line 1760, in __get__
File "mercurial\localrepo.pyc", line 1783, in dirstate
AttributeError: 'gitdirstate' object has no attribute 'refresh'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "hg", line 58, in <module>
File "mercurial\dispatch.pyc", line 143, in run
File "mercurial\dispatch.pyc", line 232, in dispatch
File "mercurial\dispatch.pyc", line 307, in _rundispatch
File "mercurial\ui.pyc", line 2172, in log
File "mercurial\loggingutil.pyc", line 142, in log
File "hgext\blackbox.pyc", line 130, in log
File "hgext\blackbox.pyc", line 147, in _log
File "mercurial\localrepo.pyc", line 1947, in __getitem__
File "mercurial\extensions.pyc", line 532, in closure
File "hgext3rd\topic\__init__.pyc", line 812, in wrapinit
File "mercurial\context.pyc", line 1562, in __init__
File "mercurial\localrepo.pyc", line 223, in __get__
File "mercurial\util.pyc", line 1760, in __get__
File "mercurial\localrepo.pyc", line 1783, in dirstate
AttributeError: 'gitdirstate' object has no attribute 'refresh'
```
This was an old clone I had laying around from 2020, so IDR how I created it (with git, hg-git, or hgext.git), but it now has both a `.git` and a `.hg` directory. The last commit locally is:
```
changeset: 8155:0e2004f6fbe9
bookmark: master
tag: tip
user: Harold Dost <h.dost@criteo.com>
date: Mon May 04 12:49:45 2020 +0200
summary: Simplify the Getting Started documentation. (#7193)
```
I suspect it's an hg 6.4 compat issue though if `dirstate.refresh` is missing.https://foss.heptapod.net/mercurial/hg-git/-/issues/408setup docs unhelpful2023-11-06T20:01:17ZRick Kilgoresetup docs unhelpfulThe following text from the project's README.rst file needs more detail imo:
```
And enable it from somewhere in your ``$PYTHONPATH``::
[extensions]
hggit =
```
I found another site (https://hg-git.github.io/) that seems to be sa...The following text from the project's README.rst file needs more detail imo:
```
And enable it from somewhere in your ``$PYTHONPATH``::
[extensions]
hggit =
```
I found another site (https://hg-git.github.io/) that seems to be saying this should be put in ~/.hgrc? Or maybe that is old info and I need to create a file with any name in my PYTHONPATH? Either way, I'm going to have to do some guessing, and a little additional info here could prevent that.
Also, the https://hg-git.github.io/ site suggested that maybe the text that goes in whatever file should be holding these settings should actually be this:
```
[extensions]
hgext.bookmarks =
hggit = [path-to]/hg-git/hggit
```
Which is correct or better, or can we have some text here describing why you might choose one over the other?https://foss.heptapod.net/mercurial/hg-git/-/issues/407Slow cloning with hg-git: an attempt at profiling2023-05-04T10:31:48ZmuxatorSlow cloning with hg-git: an attempt at profilingHi,
cloning a big git repository via hg-git has always been not really fast. For example, on my machine cloning https://github.com/bitcoin/bitcoin takes ~30 minutes.
I really hope this could be made faster; to start this effort, I thou...Hi,
cloning a big git repository via hg-git has always been not really fast. For example, on my machine cloning https://github.com/bitcoin/bitcoin takes ~30 minutes.
I really hope this could be made faster; to start this effort, I thought that doing some profiling work could help.
The bitcoin repository is too big, so I choose another project (https://github.com/kdave/btrfs-progs: 6500 commits, 148 tags, 16 MB of contents in its head revision, a .git folder of 19 MB).
First of all, the baselines:
|command|time|%baseline|
|---|---|---|
|`git clone https://github.com/kdave/btrfs-progs`|3.2 s | 100% |
|`hg clone git+https://github.com/kdave/btrfs-progs`|52.9 s | 1653% (**16 times slower**!) |
|cprofile on `hg clone git+https://github.com/kdave/btrfs-progs`|84.3 s | speed not meaningful due to the measuring overhead |
A cloning that takes ~3 seconds with git takes ~53 seconds with mercurial/hg-git: 16 times as much!
These tests were made in this environment:
- fedora 37 (kernel 6.2)
- git **v2.40.1**
- python **v3.11.3**
- hg **v6.2.3**
- hg-git **v1.0.2**
- dulwich **v0.21.3**
Hardware:
- Xeon E3-1265L V2 @ 2.50GHz (not really a bottleneck here)
- 16 GB RAM
- SATA3 SSD (not really a bottleneck here)
# profiling using cProfile
I have used cProfile to profile the cloning. The profiling overhead is significant (we go from from 53s to 84s), hopefully this will decrease when we can use perf on python3.12.
```
time python -m cProfile -o ~/btrfs-progs-clone.cprof /usr/bin/hg clone git+https://github.com/kdave/btrfs-progs
destination directory: btrfs-progs
importing 6699 git commits
new changesets 3fe272fcfd57:8a107059ab38 (6699 drafts)
updating to bookmark master
791 files updated, 0 files merged, 0 files removed, 0 files unresolved
real 1m24,293s
user 1m18,146s
sys 0m2,480s
```
The result of the profiling is here: [btrfs-progs-clone.cprof](/uploads/c32addcc8d5b4d3ab5eb397e79af2d06/btrfs-progs-clone.cprof). It can be visualized, for example, with [SnakeViz](https://jiffyclub.github.io/snakeviz/) (`pip install snakeviz`):
```
snakeviz btrfs-progs-clone.cprof
```
As said, in this thest the total time for the cloning was ~84 seconds. This is a zoom on the first meaningful function that accounts for ~100% of that time. A breakdown of the deeper functions is in the image, while the list of the previous call stack elements is on the right, for completeness:
![image](/uploads/4d581a3a970d25cb8aef456541267643/image.png)
# profiling using strace
I've also tried using strace to profile the cloning. The overhead is still higher, however.
```
time strace --summary -o ~/btrfs-progs-strace.log /usr/bin/hg clone git+https://github.com/kdave/btrfs-progs
destination directory: btrfs-progs
importing 6699 git commits
new changesets 3fe272fcfd57:8a107059ab38 (6699 drafts)
updating to bookmark master
791 files updated, 0 files merged, 0 files removed, 0 files unresolved
real 3m52,715s
user 2m5,755s
sys 1m36,003s
```
I believe the very high sys time is strace's overhead. In the plain hg-git clone the system time was just 4% of the total time (less than 3 seconds). This aligns nicely with the absolute times that strace produces in its summary:
```
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ------------------
26,91 0,889219 1 548434 3 lseek
24,04 0,794304 2 350410 read
12,80 0,422899 4 101525 17551 openat
12,53 0,414054 2 150241 2878 newfstatat
6,02 0,198903 6027 33 3 wait4
5,62 0,185728 2 83999 close
4,96 0,163822 1 87659 83051 ioctl
4,12 0,136191 3 41115 write
1,02 0,033586 5 6502 utimensat
```
I'd say that not too much time is spent in the system calls. Maybe there is a tad too much disk activity but nothing stands out.https://foss.heptapod.net/mercurial/hg-git/-/issues/399Pushing to github too long for large repository2023-01-15T09:56:49ZCédric Kriercedric.krier@b2ck.comPushing to github too long for large repositoryWe mirror https://foss.heptapod.net/tryton/tryton on GitHub https://github.com/tryton/tryton but very often the task fails because GitHub stop the connection because it takes too much time (even if there are only very few new commits).
I...We mirror https://foss.heptapod.net/tryton/tryton on GitHub https://github.com/tryton/tryton but very often the task fails because GitHub stop the connection because it takes too much time (even if there are only very few new commits).
Is there a way to speed up the push? We already run `hg gexport` before the push but it is still not enough. I suspect it is the number of tags (>9000) that takes so much time.https://foss.heptapod.net/mercurial/hg-git/-/issues/395New mode to use named branches and topics instead of bookmarks2022-12-28T07:46:03ZPierre AugierNew mode to use named branches and topics instead of bookmarksThis is a very long term issue.
I had to work a lot on a project using few Git repositories on Github. hg-git works well for this task but the user experience with bookmarks is not as good as for a standard hg repo hosted for example on...This is a very long term issue.
I had to work a lot on a project using few Git repositories on Github. hg-git works well for this task but the user experience with bookmarks is not as good as for a standard hg repo hosted for example on foss.heptapod.net. For example:
- No changesets are published even though we have protected branches on the repos that could be considered as "publishing".
- In some cases, `hg up` should not be used without option because it can move the activated bookmark where it should not be moved.
- `hg stack` does not work as expected (it shows all changesets).
- I can't use `hg absorb` because it does not understand that only the changesets in one Git branch can be modified.
- `hg rebase` (without option) does not automatically detect what has to be done.
- commits in deleted Git branches are not hidden
- commits that should be hidden by evolve commands are not hidden
- It is sometimes difficult to see from which feature branch a commit comes from
Some of these problems comes from the fact that hg-git uses bookmarks, which are of course very similar to Git branches but which do not interact very well with other Mercurial commands.
It seems to me that hg-git could have a mode using named branched (restricted to 1 head) for protected git branches and topics (restricted to 1 head) for other Git branches (for feature branches). This would make hg with hg-git a very attractive client to work with Git repositories. Therefore, having this mode could have the potential to change how a lot of users interact with Github/Gitlab.
I'd like to ask about the technical feasibility of such mode?https://foss.heptapod.net/mercurial/hg-git/-/issues/390Dulwich installed but not found2022-12-23T15:29:07ZKen ArnoldDulwich installed but not foundI have used pip3 to install hggit, which reports that the requirements are satisfied, including dulwich. But when I run hg commands (hg push, for example), I get the error
`*** failed to import extension "hggit" from /Users/kcrca/homebr...I have used pip3 to install hggit, which reports that the requirements are satisfied, including dulwich. But when I run hg commands (hg push, for example), I get the error
`*** failed to import extension "hggit" from /Users/kcrca/homebrew/lib/python3.9/site-packages/hggit: No module named 'dulwich'`
(Yes, python is installed via homebrew in a personal directory, but that has never been a problem before.)
I have tried various locations for hggit, and several other things to try to fix it, but it won't budge. And I can't find any advice on this online. FWIW, dulwich is in the same site-packages directory:
```
% ls -ld /Users/kcrca/homebrew/lib/python3.9/site-packages/hggit /Users/kcrca/homebrew/lib/python3.9/site-packages/dulwich
drwxrwxr-x 48 kcrca primarygroup 1536 Jul 16 14:46 /Users/kcrca/homebrew/lib/python3.9/site-packages/dulwich
drwxrwxr-x 21 kcrca primarygroup 672 Jul 16 15:27 /Users/kcrca/homebrew/lib/python3.9/site-packages/hggit
```
Any ideas?https://foss.heptapod.net/mercurial/hg-git/-/issues/388Error pushing to GitHub on python3.102022-07-11T18:00:46ZSanjay VasandaniError pushing to GitHub on python3.10Our managed Linux workstations were just upgraded to python3.10. Now pushing to GitHub throws an error. Partial stack trace below, with a more complete log [attached](/uploads/c984810c55cfd28cef02be8756fa6049/hg.log).
I'm not certain if...Our managed Linux workstations were just upgraded to python3.10. Now pushing to GitHub throws an error. Partial stack trace below, with a more complete log [attached](/uploads/c984810c55cfd28cef02be8756fa6049/hg.log).
I'm not certain if the issue is in hg-git, dulwich, or urllib3. I'm also not certain if this is specific to GitHub.
```
Traceback (most recent call last):
File "/usr/lib/python3.10/http/client.py", line 565, in _get_chunk_left
chunk_left = self._read_next_chunk_size()
File "/usr/lib/python3.10/http/client.py", line 532, in _read_next_chunk_size
return int(line, 16)
ValueError: invalid literal for int() with base 16: b''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.10/http/client.py", line 582, in _read_chunked
chunk_left = self._get_chunk_left()
File "/usr/lib/python3.10/http/client.py", line 567, in _get_chunk_left
raise IncompleteRead(b'')
http.client.IncompleteRead: IncompleteRead(0 bytes read)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/response.py", line 441, in _error_catcher
yield
File "/usr/lib/python3/dist-packages/urllib3/response.py", line 522, in read
data = self._fp.read(amt) if not fp_closed else b""
File "/usr/lib/python3.10/http/client.py", line 459, in read
return self._read_chunked(amt)
File "/usr/lib/python3.10/http/client.py", line 597, in _read_chunked
raise IncompleteRead(b''.join(value))
http.client.IncompleteRead: IncompleteRead(0 bytes read)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/hg", line 59, in <module>
dispatch.run()
File "/usr/lib/python3/dist-packages/mercurial/dispatch.py", line 143, in run
status = dispatch(req)
```https://foss.heptapod.net/mercurial/hg-git/-/issues/385Non-publishing remotes can lead to confusing results2022-04-27T16:56:18ZErnie RaelNon-publishing remotes can lead to confusing resultsThe old clone shows some obsolete changeset that are actually part of the mainline on github. I re-cloned the github repository and things look OK. Not sure, but probably probably started after an `hg pull`.
```
$ hg --version -v
Mercuri...The old clone shows some obsolete changeset that are actually part of the mainline on github. I re-cloned the github repository and things look OK. Not sure, but probably probably started after an `hg pull`.
```
$ hg --version -v
Mercurial Distributed SCM (version 6.1)
hggit external 1.0.0 (dulwich 0.20.35)
```
The issue (AFAICT) can be seen here
**OLD clone:**
```
| x | 1100:c3f27bcff8c6:4f55878589be draft 4.0.9-SNAPSHOT setup.
x | 1098:a1739e837eca:76ed80f9a352 draft add SSComponentInterface.getSSDataNavigator()
```
**NEW clone:**
```
| o 1087:c3f27bcff8c6:4f55878589be draft 4.0.9-SNAPSHOT setup.
o 1086:a1739e837eca:76ed80f9a352 draft add SSComponentInterface.getSSDataNavigator()
```
Note that the corresponding items have identical gitnode changeset IDs.
And anything descending from these changesets are marked unstable. Very disconcerting :-)
Following shows the changesets in context. Note that in the OLD-CLONE, the "obsolete" changesets were involved in a rebase acc'd to the log message. (there _was_ some ugly shuffling in this area).
Unfortunatly, I don't know if the changesets were marked obsolete before I did the pull. This clone was migrated to a new system several weeks ago and this is the first time it's used; started with a pull.
NEW-CLONE
```
| o | 1089:7f3963c008c5:73d2eb898de6 draft Clean up some eclipse warnings
| | |
| o | 1088:23c664bc885b:34c5dcb45705 draft Comment historical context for getSSDataNavigator methods
| |/
| o 1087:c3f27bcff8c6:4f55878589be draft 4.0.9-SNAPSHOT setup.
|/
o 1086:a1739e837eca:76ed80f9a352 draft add SSComponentInterface.getSSDataNavigator()
|
o 1085:de6e26488956:75a5891c38b3 draft Removing -SNAPSHOT for 4.0.8 release.
```
OLD-CLONE
```
| * | 1107:7f3963c008c5:73d2eb898de6 draft Clean up some eclipse warnings
| | |
| * | 1106:23c664bc885b:34c5dcb45705 draft Comment historical context for getSSDataNavigator methods
| |/
| | o 1105:df0e5c4ca471:465f29f6c3d6 draft Fix SSFormattedTextField constructors to use this() to avoid NPE
| | |
| | o 1104:0da71ca91ed9:4df66a5c42cc draft SSMaskFormatter and some UI experiments
| | |
| | o 1103:b52ff62e38ba:295824736b95 draft 4.0.9-SNAPSHOT setup.
| | |
| x | 1100:c3f27bcff8c6:4f55878589be draft 4.0.9-SNAPSHOT setup.
|/ /
| o 1099:b625af0e54b3:7d253d10187d draft add SSComponentInterface.getSSDataNavigator()
| |
x | 1098:a1739e837eca:76ed80f9a352 draft add SSComponentInterface.getSSDataNavigator()
|/
| o 1097:9b7606c645c0:dded08d8493e draft Unbonking the CHANGLOG for 4.0.9-SNAPSHOT branch.
| |
| o 1093:be5bbd718cdc:2af40657fd26 draft Setup for 4.0.9-SNAPSHOT branch.
|/
o 1091:de6e26488956:75a5891c38b3 draft Removing -SNAPSHOT for 4.0.8 release.
```
```
| x | changeset: 1100:c3f27bcff8c6
|/ / parent: 1098:a1739e837eca
| | user: Brian Pangburn <swingset@pangburngroup.com>
| | date: Fri Feb 11 18:15:19 2022 -0600
| | obsolete: rebased using evolve as 1103:b52ff62e38ba
| | summary: 4.0.9-SNAPSHOT setup.
| |
| o changeset: 1099:b625af0e54b3
| | parent: 1091:de6e26488956
| | user: Ernie Rael <errael@raelity.com>
| | date: Fri Feb 11 22:44:26 2022 +0000
| | summary: add SSComponentInterface.getSSDataNavigator()
| |
x | changeset: 1098:a1739e837eca
|/ parent: 1091:de6e26488956
| user: Ernie Rael <errael@raelity.com>
| date: Fri Feb 11 22:44:26 2022 +0000
| obsolete: amended using amend as 1099:b625af0e54b3
| summary: add SSComponentInterface.getSSDataNavigator()
```https://foss.heptapod.net/mercurial/hg-git/-/issues/382Bookmarks do not get updated on pull if remote branch was force-pushed2022-03-14T21:36:29ZSanjay VasandaniBookmarks do not get updated on pull if remote branch was force-pushedI believe this applies only to the 1.0.0 beta.
The logic for detecting whether it's safe to update a bookmark on pull seems to be incorrect in the case that the remote branch was force-pushed. In this case, hg-git fails to update the bo...I believe this applies only to the 1.0.0 beta.
The logic for detecting whether it's safe to update a bookmark on pull seems to be incorrect in the case that the remote branch was force-pushed. In this case, hg-git fails to update the bookmark and indicates that it's diverged.
What I'd expect is that hg-git checks the bookmark against the old changeset for the Git ref prior to pull. If it's the same, then it should be safe to move the bookmark to the new changeset for the Git ref. What hg-git appears to be doing instead is trying to determine whether the Git branch was fast-forwarded.https://foss.heptapod.net/mercurial/hg-git/-/issues/377Command to run git with git-dir2021-12-24T20:44:23ZCarl ReinkeCommand to run git with git-dirIt would be handy to have a command that runs `git` with the right git-dir and work-tree. (Of course, this is only really useful when `git.intree` is `False`.)
The command would do something like this:
`git --git-dir="$hgroot/.hg/git" ...It would be handy to have a command that runs `git` with the right git-dir and work-tree. (Of course, this is only really useful when `git.intree` is `False`.)
The command would do something like this:
`git --git-dir="$hgroot/.hg/git" --work-tree="$hgroot" "$@"`
So `hg git push origin master` would run `git --git-dir="/path/to/.hg/git" --work-tree="/path/to" push origin master`.https://foss.heptapod.net/mercurial/hg-git/-/issues/374a strange tag is added when pushing to non master branches2021-12-24T12:42:25ZUwe Brauera strange tag is added when pushing to non master branches
o...@ucm.es
Dec 14, 2021, 4:41:22 PM (16 hours ago)
to hg-git
Hi
I am currently using the changeset: 4b51ff13e8f5
With the topic branches-and-stuff
hg clone git+ssh://o...@git.code.sf.net/p/matlab-emacs/src
I created a named bran...
o...@ucm.es
Dec 14, 2021, 4:41:22 PM (16 hours ago)
to hg-git
Hi
I am currently using the changeset: 4b51ff13e8f5
With the topic branches-and-stuff
hg clone git+ssh://o...@git.code.sf.net/p/matlab-emacs/src
I created a named branch, edit commit and pushed to bitbucket sandbox
I cloned that sandbox with git:
git clone g...@bitbucket.org:kalthad/emacs-matlab-sandbox.git
Then
git log --all shows me:
```
* commit 6e05cc2502ad1ce5dbd278233178a122f6995b82 (origin/copyright)
[-]
| Author: Uwe Brauer <o...@mat.ucm.es>
| Date: Sat Dec 11 18:09:35 2021 +0100
|
| New file, to check copyright issues
|
| * copyright.org (Patches: checked so far): 678 New file, that
| contains a list of patches and whether the author's signature
| might be needed.
|
| --HG--
| branch : copyright
```
However when I a changeset to the default
branch, pushed and then pulled with git.
As expected the changeset ended up in the git's master branch but
without that TAG
```
| * commit 33dd1074de31d838a8e27fcbe5f93ba2a0a54142 (HEAD -> master, origin/master, origin/HEAD)
| | Author: Uwe Brauer <o...@mat.ucm.es>
| | Date: Sun Dec 12 08:11:51 2021 +0100
| |
| | Remove patches and copyright.org
| |
| | * .hgignore (syntax): 678 remove patches, remove copyright.org
| |
| | * commit 6e05cc2502ad1ce5dbd278233178a122f6995b82 (origin/copyright)
| |/ Author: Uwe Brauer <o...@mat.ucm.es>
| | Date: Sat Dec 11 18:09:35 2021 +0100
| |
| | New file, to check copyright issues
| |
| | * copyright.org (Patches: checked so far): 678 New file, that
| | contains a list of patches and whether the author's signature
| | might be needed.
| |
| | --HG--
| | branch : copyright
| |
`----
```
So I am puzzled. Can anybody tell me whats going on here?
Regards
Uwe Brauer2.0.0https://foss.heptapod.net/mercurial/hg-git/-/issues/372the topic branches-and-stuff: some complex git repos are not cloned correctly2021-12-24T12:44:06ZUwe Brauerthe topic branches-and-stuff: some complex git repos are not cloned correctlyHi
I cloned
```
git@github.com:kjambunathan/org-mode-ox-odt.git
```
and run hg out
and obtained:
```
comparing with git@github.com:kjambunathan/org-mode-ox-odt.git
warning: not overwriting annotated tag 'release_8.3beta'
warning: not ov...Hi
I cloned
```
git@github.com:kjambunathan/org-mode-ox-odt.git
```
and run hg out
and obtained:
```
comparing with git@github.com:kjambunathan/org-mode-ox-odt.git
warning: not overwriting annotated tag 'release_8.3beta'
warning: not overwriting annotated tag 'release_5.23a'
warning: not pushing branch 'default' as it has multiple heads
Warning: Permanently added 'github.com,140.82.121.3' (ECDSA) to the list of known hosts.
```
so something did not work.
The repository in question is rather complex
the one I maintain
git+ssh://oub@git.code.sf.net/p/matlab-emacs/src
was cloned correctly2.0.0https://foss.heptapod.net/mercurial/hg-git/-/issues/371the branches-and-stuff topic, hcangest: 6942fb0a8c9d: can't push default bran...2021-12-24T12:44:15ZUwe Brauerthe branches-and-stuff topic, hcangest: 6942fb0a8c9d: can't push default branch with various heads (although they have bookmarks)I am currently trying out your branch/topic
1874:4b51ff13e8f5
I encountered a problem
Here is what I am doing. I am maintain matlab-emacs, and I used hg-git but with Manuel Jacobs patch
```
◍ changeset: 1616:6942fb0a8c9d
│ Br...I am currently trying out your branch/topic
1874:4b51ff13e8f5
I encountered a problem
Here is what I am doing. I am maintain matlab-emacs, and I used hg-git but with Manuel Jacobs patch
```
◍ changeset: 1616:6942fb0a8c9d
│ Branch: default
│ Author: Manuel Jacob <me@manueljacob.de>
│ Date: Sat, 01 Jun 2019 16:54:53 +0200
│ Topic: export-additional-refs
│ Phase: draft
│ Summary: git_handler: add mechanism to specify revsets to be exported to Git refs according to a template
│
○ changeset: 1615:cd663e6379f0
│ Branch: default
│ Author: Manuel Jacob <me@manueljacob.de>
│ Date: Sat, 01 Jun 2019 16:55:10 +0200
│ Topic: same-ref-for-multiple-changesets
│ Phase: draft
│ Summary: git_handler: make sure that the same ref is not used for multiple changesets
```
Maybe I shouldn't have done that
Now I tried out your topic branch
``` changeset: 1874:4b51ff13e8f5
│ Branch: default
│ Author: Dan Villiom Podlaski Christiansen <danchr@gmail.com>
│ Date: Sat, 08 May 2021 19:45:01 +0200
│ Topic: branches-and-stuff
│ Phase: draft
│ Summary: modes: add support for topics
│
```
With hg 5.5
So here is what I did in a temporary directory:
- hg clone git+ssh://oub@git.code.sf.net/p/matlab-emacs/src
- mv src matlab-mode
- cd matlab-mode
- create a empty git repository in bitbucket (can be any repositories)
- I run hg push
But then I receive:
pushing to git@bitbucket.org:kalthad/emacs-matlab-sand2.git
warning: not pushing branch 'default' as it has multiple heads
Warning: Permanently added 'bitbucket.org,2406:da00:ff00::22e9:9f55' (RSA) to the list of known hosts.
searching for changes
adding objects
added 450 commits with 601 trees and 600 blobs
adding reference refs/heads/documentation
Warning: Permanently added 'bitbucket.org,2406:da00:ff00::22e9:9f55' (RSA) to the list of known hosts.
That is a problem the default branch has head with bookmarks these have
been generated when importing all the git branches.
So the push did leave out a considerable part of my repository.
Any idea what to do? I don't want to merge these heads, because it might
confuse the git users I am interactive with.
Maybe using Manuel's patch somehow damaged the repository in the sens
that it creates to many heads.
Any help is welcome
Regards
Uwe2.0.0https://foss.heptapod.net/mercurial/hg-git/-/issues/370can hggit clone hystory partial - last N records?2021-11-15T16:35:25Zalexraynecan hggit clone hystory partial - last N records?Hallow, i intends work with git repo with more 60K hystory records. Is there any way to clone into mercurial just last 10k? can hggit do it?Hallow, i intends work with git repo with more 60K hystory records. Is there any way to clone into mercurial just last 10k? can hggit do it?https://foss.heptapod.net/mercurial/hg-git/-/issues/367Branches that are deleted upstream can't be deleted locally.2022-04-03T15:01:15ZjmbBranches that are deleted upstream can't be deleted locally.In our company the `master` branch is protected and every contribution must be done to a temporary branch and go through a merge request and review before it is integrated. This results in a lot of temporary branches that get deleted on ...In our company the `master` branch is protected and every contribution must be done to a temporary branch and go through a merge request and review before it is integrated. This results in a lot of temporary branches that get deleted on the server when they are merged. However those branches are not deleted locally, and I haven't found a clean way to delete them:
- For an obsolete branch `foo`, I can delete the `foo` bookmark but not the `default/foo` "tag" that corresponds to the upstream branch.
- If I strip the local branch, it gets recreated the next time I pull even though it no longer exists on the server! I assume it was kept in the local git cache?
- The only way I've found to get rid of the deleted branches is to either run `hg gclear` or `rm .hg/git/refs/heads/* .hg/git/refs/remotes/default/*`, then the next `pull` restores only the branches that still exist upstream and I can strip the stale branches.
Note that developpers who use git directly have no problem because `git pull` only fetches the current branch, whereas `hg pull` fetches all existing branches (see also #357). So they don't see the branches from other's merge requests.Dan Villiom Podlaski Christiansendanchr@gmail.comDan Villiom Podlaski Christiansendanchr@gmail.com