gitifyhg icon indicating copy to clipboard operation
gitifyhg copied to clipboard

Attempting to clone pypy repo failed for me.

Open kcr opened this issue 12 years ago • 15 comments

... progress revision 61099 'master' (61100/62605) fatal: Empty path component found in input fast-import: dumping crash report to /home/kcr/src/pypy/.git/fast_import_crash_8317 fatal: Error while running fast-import kcr@ordinator$ Traceback (most recent call last): File "/usr/bin/git-remote-gitifyhg", line 9, in load_entry_point('gitifyhg==0.8', 'console_scripts', 'git-remote-gitifyhg')() File "/usr/lib/python2.7/dist-packages/gitifyhg/gitifyhg.py", line 262, in main HGRemote(*[x.decode('utf-8') for x in sys.argv[1:3]]).process() File "/usr/lib/python2.7/dist-packages/gitifyhg/gitifyhg.py", line 172, in process getattr(self, 'do_%s' % command)(parser) File "/usr/lib/python2.7/dist-packages/gitifyhg/gitifyhg.py", line 252, in do_import HGImporter(self, parser).process() File "/usr/lib/python2.7/dist-packages/gitifyhg/hgimporter.py", line 87, in process self.hgremote.headnode[1]) File "/usr/lib/python2.7/dist-packages/gitifyhg/hgimporter.py", line 209, in process_ref output(data) File "/usr/lib/python2.7/dist-packages/gitifyhg/util.py", line 40, in output print >> actual_stdout, msg IOError: [Errno 32] Broken pipe

command was

git clone gitifyhg::ssh://[email protected]/pypy/pypy

gitifyhg from the 0.8 tag, git version 1.7.10.4, hg 2.2.2

kcr avatar Mar 21 '13 05:03 kcr

Thanks for the report. This used to work, so it looks like a regression. In any case, we definitely should fix this.

fingolfin avatar Mar 22 '13 16:03 fingolfin

Works fine in remote-hg:

https://github.com/felipec/git/commit/a160978911801114c781efb15973c26e0b8b9822

felipec avatar Apr 05 '13 22:04 felipec

After observing that #69 did not fix this, and that the master of git-remote-hg doesn't work either, I noticed the commit felipec/git@a160978 is intended to fix this problem. I've adapted that commit to gitifyhg. Thanks, Felipe!

PaulPrice avatar Apr 15 '13 18:04 PaulPrice

One thing I don't like about github is how commenting on commits and pull requests are separated. I meant to put this here:

Hi Paul,

You're doing a wonderful job of making me feel productive! ;) I thought I was going to do this on Sunday but it didn't happen.

I'm merging this patch as is but I'd like to point out that we should not have to call os.path.relpath here, because path is a path.py object which has a relpath method. However, path.path.relpath() does not accept the 'start' parameter, so we can't use it. I've filed jaraco/path.py#20 on path.py for this.

Your next pull request should be an update to add your name to the Readme as a contributor. Though, if you keep it up you'll have to move it to maintainers pretty quickly. :-)

I think we'll be calling 0.8.2 the Paul Price release!

dusty-phillips avatar Apr 15 '13 19:04 dusty-phillips

I use gitifyhg for my work (to avoid using Mercurial), so I'm happy to be able to contribute to improving it. I'm looking forward to 0.8.2, but please let's not call it anything silly.

OK, my attempt to clone pypy just now failed (but it got further than before). Will paste the traceback from my desktop....

PaulPrice avatar Apr 15 '13 19:04 PaulPrice

progress revision 62999 'master' (63000/63382)
progress revision 63099 'master' (63100/63382)
progress revision 63199 'master' (63200/63382)
progress revision 63299 'master' (63300/63382)
WARNING: "Branch 'dist' has more than one head, consider merging"
[...]
WARNING: "Branch 'unicode-objspace' has more than one head, consider merging"
WARNING: "Branch 'reflex-support' has more than one head, consider merging"
error: there are still refs under 'refs/hg/origin/branches/stdlib-unification'
error: Unable to lock refs/hg/origin/branches/stdlib-unification
fatal: Error while running fast-import
price@neverland:~/test $ Traceback (most recent call last):
  File "/home/price/Software/gitifyhg/bin/git-remote-testgitifyhg", line 19, in <module>
    gitifyhg.main()
  File "/home/price/Software/gitifyhg/gitifyhg/gitifyhg.py", line 321, in main
    HGRemote(*[x.decode('utf-8') for x in args]).process()
  File "/home/price/Software/gitifyhg/gitifyhg/gitifyhg.py", line 197, in process
    self.marks.store()
  File "/home/price/Software/gitifyhg/gitifyhg/util.py", line 197, in store
    with self.storage_path.open('w') as file:
  File "/home/price/local/Linux.x86_64/lib/python2.7/site-packages/path.py-3.0.1-py2.7.egg/path.py", line 589, in open
    return open(self, mode)
IOError: [Errno 2] No such file or directory: path(u'/home/price/test/pypy/.git/hg/bc0c25890da4851631fef4c9786192c2b5377b6d/marks-hg')

PaulPrice avatar Apr 15 '13 19:04 PaulPrice

Ugh, wouldn't it be nice to be able to test this without waiting half an hour for pypy to clone? At least it's a good repo for uncovering problems.

The IOError is a red herring; that's showing up because git has deleted the failed repo by the time gitifyhg tries to update the marks. Really, marks should only be updated if the command was successful.

The unable to lock error is a new one to me and a mystery. Runing with DEBUG_GITIFYHG may yield some more insight. There's clearly something funny going on with the git-remote protocol. However, cloning pypy with debugging output enabled may take a day to complete. It would be ideal to come up with a unit test that reproduces the issue.

dusty-phillips avatar Apr 15 '13 21:04 dusty-phillips

Started the run with DEBUG_GITIFYHG; will check it in the morning. (:

PaulPrice avatar Apr 15 '13 21:04 PaulPrice

The output is 11GB! I'm not going to post it all here. (: But here's a sample.

[...]
WARNING: "Branch 'stdlib-unification/py3k' has more than one head, consider merging"
DEBUG: 'OUT: ? refs/heads/branches/stdlib-unification/py3k'
[...]
DEBUG: 'OUT: ? refs/heads/branches/stdlib-unification'
[...]
DEBUG: 'INPUT: import refs/tags/pypy-2.0-beta2'
DEBUG: 'OUT: reset refs/hg/origin/tags/pypy-2.0-beta2'
DEBUG: 'OUT: from :63129'
DEBUG: 'OUT: '
DEBUG: 'INPUT: '
DEBUG: 'OUT: done'
error: there are still refs under 'refs/hg/origin/branches/stdlib-unification'
error: Unable to lock refs/hg/origin/branches/stdlib-unification
fatal: Error while running fast-import
DEBUG: 'INPUT: '
Traceback (most recent call last):
  File "/home/price/Software/gitifyhg/bin/git-remote-testgitifyhg", line 19, in <module>
    gitifyhg.main()
  File "/home/price/Software/gitifyhg/gitifyhg/gitifyhg.py", line 321, in main
    HGRemote(*[x.decode('utf-8') for x in args]).process()
  File "/home/price/Software/gitifyhg/gitifyhg/gitifyhg.py", line 197, in process
    self.marks.store()
  File "/home/price/Software/gitifyhg/gitifyhg/util.py", line 197, in store
    with self.storage_path.open('w') as file:
  File "/home/price/local/Linux.x86_64/lib/python2.7/site-packages/path.py-3.0.1-py2.7.egg/path.py", line 589, in open
    return open(self, mode)
IOError: [Errno 2] No such file or directory: path(u'/home/price/test/pypy/.git/hg/c58fbfe850579675ea5d00a2a177d7b514440104/marks-hg')

It seems to me that this is likely related to the existence of a branch stdlib-unification/py3k in addition to branch stdlib-unification.

PaulPrice avatar Apr 16 '13 13:04 PaulPrice

You're probably right, I think there's a couple evil lines in gitifyhg that split on '/'. The best way to test this would be to write a unit test with branches that share a prefix and have a '/'. No sense cloning pypy with 11 GB of output (!!!!) every time ;-)

dusty-phillips avatar Apr 16 '13 14:04 dusty-phillips

It seems to me that this is likely related to the existence of a branch stdlib-unification/py3k in addition to branch stdlib-unification.

Yes, git does not allow this.

$ git branch x
$ git branch x/y
error: unable to resolve reference refs/heads/x/y: Not a directory
fatal: Failed to lock ref for update: Not a directory

Git branch names are like directory names, and literally exist in directories until they are written to the pack file.

jedbrown avatar Apr 16 '13 16:04 jedbrown

Bummer. That disproves my comments on #75 I'd still like to find a neater solution for this than replacing / with SLASH, though. any ideas? Even if we only replaced it in the case where a common prefix is already known to exists, but the thought of maintaining that scares me.

So in git, having this naming structure is legal:

feature/hello
feature/world

but having this structure is not legal

feature
feature/hello

Is there some way we can do substitution only in the second case when we know that an error is going to occur? I'm thinking not, because if feature/hello exists and feature is added later, we have a different problem than if feature exists first and feature/hello is added later.

Even using a slightly less ugly substitution than SLASH would make me feel more content, but I don't know what to suggest. SLASH is at least explicit and not likely to conflict with other apis.

dusty-phillips avatar Apr 16 '13 17:04 dusty-phillips

This was pointed out by Junio http://article.gmane.org/gmane.comp.version-control.git/220034. Which shows why patch review by git developers is valuable.

felipec avatar Apr 16 '13 18:04 felipec

I haven't tested it on pypy yet, but I suspect the workaround for #80 may address this issue. It depends if either of the conflicting branches is a closed branch in hg or not.

dusty-phillips avatar Apr 30 '13 21:04 dusty-phillips

I believe it works in remote-hg because it doesn't clone the closed branches, so I expect gitifyhg would work now too.

PaulPrice avatar Apr 30 '13 21:04 PaulPrice