"ERROR: unhandled export command: mark :4" upon push

[alexei-31]

On Windows push into hg repository crash if it was cloned and has some commits. I use next script "run_test.cmd" to clone and push (Download it):

if exist "./test5/run_test.cmd" rd /S /Q "./test5"
git clone https://github.com/alexei-31/test5.git
@cd test5
git push hg::https://bitbucket.org/Alexei_31/test5
@cd ..

As result test script "run_test.cmd" generate next error:

Also, I can give the file to the folder ".git", obtained by the program on my computer with OS Windows: Download test5.zip – ".git" folder archive in invalid state (after program crash)

Can help me anyone to find problem in program code (git-remote-hg) and/or fix this?

[alexei-31]

I made some experiments and found a minimal repository, generating this error. This script can work offline and may be used to automatize test. If it is more convenient for you, try this script "test.cmd" (Download it):

hg init test.hg
cd test.hg
echo a > a
hg add a
hg commit -m a
echo b > b
hg add b
hg commit -m b
cd ..
git clone hg::test.hg test.git
cd test.git
git push hg::../test.hg
cd ..

This creates exactly the same failure, so I will not make a screenshot again.

Download test.zip – ".git" and ".hg" folders archive in invalid state (after crash this script)

[fingolfin]

I cannot reproduce this on Mac OS X. The test script you provided completes without any error. This is with git 2.6.3 and hg 3.6.1 -- which versions are you using, esp. of git?

[alexei-31]

I am use git version 2.6.3.windows.1 and Mercurial Distributed SCM (version 3.6.1)

[alexei-31]

I think, it is specific to OS, but I save archive with "git" folder (directory). Can you get link to your result of work of "run_test.cmd" to compare with my variant?

[alexei-31]

I save archive with result of work offline mini-script on my computer, if it can help.

[fingolfin]

One immediate difference I see is that in my test.git, the remote URL is absolute: hg::/private/tmp/test.hg whereas in yours, it isn't: hg::test.hg. This is known to cause issues, and is why fix_path exists (which is not to be confused with fix_file_path, the one we already modified).

Let's see if that helps: Can you change git clone hg::test.hg test.git to something using an absolute path? Perhaps git clone hg::C:\PATH\test.hg test.git or if you are using cygwin, perhaps git clone hg::/cygdrive/c/PATH/test.hg test.git? You probably know better than me ;-).

[alexei-31]

Unfortunately, it does not help in correcting work of script "test.cmd". Moreover, the first script at this page "test5.cmd" uses only the URL-path and also has this error.

Also does not help to replace "/" (slash) by "\" (backslash) in style old OS MS-DOS. As I mentioned, Windows does not make a distinction between them in most cases.

[fingolfin]

OK (though fixing fix_file_path for Windows still would be a good idea, but apparently won't help with this issue).

[fingolfin]

So in my test.git/.git/hg/origin/marks-git I see 5 marks, in yours there are only 3 marks, which might explain the error on mark 4. That's weird.

Aha, moreover, your test.git (from your test.zip) only contains the file "a", while mine also has "b". That looks as if the clone somehow failed to import everything into git. That's even weirder.

[alexei-31]

Yes, it weird. Which experiment can help to solve it? I can modify git-remote-hg for it (add logs?), use any repositories for tests and etc. As I understand it, some "mark" are not created to commit, which are made through commands git, but successfully created via the pull and clone commands.

[alexei-31]

There is also a problem if you delete the folder ".git / hg" and let the program create it again. Author felipec wrote tool to check and fix git-remote-hg marks git-marks-check, but I has no ruby on my OS. Even if this utility will fix everything, а program error has exactly also.

[alexei-31]

As it turns out, there is an older version of Ruby 1.9.3-p 551, which is compatible with Windows XP. Unfortunately, the script does not find any problems, but the push is not working.

[fingolfin]

The git-marks-check tool is also present in this repository :). However, I don't think it applies here -- it is there to fix corruption, but the problem you are seeing are missing marks, which is not what.

Hmm... actually, what happens if you do a pull or fetch right after the clone (before you try to push) ?

[alexei-31]

Then everything is fine. Usually, errors begin, if I commit something.

[fingolfin]

So... git push gives no error? Only git push hg::../test.hg?

[alexei-31]

Yes, error push immediately after the clone of hg repository is rare in the real work. For me, it is strange that the test "test.cmd" has failed. Normally, if no commit by git commands, then the push is successful (no error mark). However, it is useless to do push, if there are no changes. The larger the repository, the greater the number mark is causing the problem. Also had cases of successful push after commit, if is "Initial commit" (even if with files in it). Or if it is a fresh new repository has never push (it needs to be checked).

[fingolfin]

@alexei-31 Sorry, again I do not quite understand what you are saying... Anyway: In your example with test5, you are cloning via https:

git-remote-hg treates clones of local repositories (same file system) differently from clones via https / ssh / etc. ! This can cause problems by itself. So, perhaps let's not mix the two. See also d006a645e4f1a91d9d8520e53548e7c78138531b and

BTW: screenshots are not so helpful. Is there a way to copy & past text from a windows terminal? Perhaps if one uses a better terminal, like http://cmder.net/ ?

[fingolfin]

@alexei-31 So suppose you run your own test script except for the last push, i.e. run this:

hg init test.hg
cd test.hg
echo a > a
hg add a
hg commit -m a
echo b > b
hg add b
hg commit -m b
cd ..
git clone hg::test.hg test.git
cd test.git

At this point, please answer:

1. What is the content of .git/hg/origin/marks-git ?
2. What does git push output?
3. What does git push hg::../test.hg output?

Thank you!

[alexei-31]

1. The content of .git/hg/origin/marks-git:

:1 d35a7fdf2328b06500f59616f2af47f1f1302a4f
:2 3b413cdf6c0eadce5353fb99e5a2451323fe1d15
:3 d165e17305bc7222dfd3ece031fdfc5534522f33
:4 cb4f8a1308ad42640e1f3ef3a1a9ed7c7308576f
:5 34afcc2adcc0eb8c17b51a13f1c1cd42d739d314

2. Output of git push:

C:\Temp\test\test.git>git push
ERROR: repository test.hg not found

3. Output of git push hg::../test.hg:

C:\Temp\test\test.git>git push hg::../test.hg
ERROR: unhandled export command: mark :4
error: failed to push some refs to 'hg::../test.hg'

[fingolfin]

Can you try setting the GIT_TRANSPORT_HELPER_DEBUG environment variable, and then try another push? E.g. on Unix, this works: GIT_TRANSPORT_HELPER_DEBUG=1 git push That should give additional debug output.

[alexei-31]

C:\Temp\test\test.git>set GIT_TRANSPORT_HELPER_DEBUG=1

C:\Temp\test\test.git>git push
Debug: Remote helper: -> capabilities
Debug: Remote helper: Waiting...
ERROR: repository test.hg not found
Debug: Remote helper quit.

[fingolfin]

Yeah, sorry, I mean of course: with the path (that a pure git push doesn't work is another bug)

[alexei-31]

And other – git push hg::../test.hg:

C:\Temp\test\test.git>set GIT_TRANSPORT_HELPER_DEBUG=1

C:\Temp\test\test.git>git push hg::../test.hg
Debug: Remote helper: -> capabilities
Debug: Remote helper: Waiting...
Debug: Remote helper: <- import
Debug: Got cap import
Debug: Remote helper: Waiting...
Debug: Remote helper: <- export
Debug: Got cap export
Debug: Remote helper: Waiting...
Debug: Remote helper: <- refspec refs/heads/branches/*:refs/hg/e309a3ed3ccfb63a4e5c2b3f6edf391a1dbef14f/branches/*
Debug: Got cap refspec refs/heads/branches/*:refs/hg/e309a3ed3ccfb63a4e5c2b3f6edf391a1dbef14f/branches/*
Debug: Remote helper: Waiting...
Debug: Remote helper: <- refspec refs/heads/*:refs/hg/e309a3ed3ccfb63a4e5c2b3f6edf391a1dbef14f/bookmarks/*
Debug: Got cap refspec refs/heads/*:refs/hg/e309a3ed3ccfb63a4e5c2b3f6edf391a1dbef14f/bookmarks/*
Debug: Remote helper: Waiting...
Debug: Remote helper: <- refspec refs/tags/*:refs/hg/e309a3ed3ccfb63a4e5c2b3f6edf391a1dbef14f/tags/*
Debug: Got cap refspec refs/tags/*:refs/hg/e309a3ed3ccfb63a4e5c2b3f6edf391a1dbef14f/tags/*
Debug: Remote helper: Waiting...
Debug: Remote helper: <- *export-marks .git\hg\e309a3ed3ccfb63a4e5c2b3f6edf391a1dbef14f\marks-git
Debug: Got cap export-marks .git\hg\e309a3ed3ccfb63a4e5c2b3f6edf391a1dbef14f\marks-git
Debug: Remote helper: Waiting...
Debug: Remote helper: <- option
Debug: Got cap option
Debug: Remote helper: Waiting...
Debug: Remote helper: <-
Debug: Capabilities complete.
Debug: Remote helper: -> option progress true
Debug: Remote helper: Waiting...
Debug: Remote helper: <- unsupported
Debug: Remote helper: -> option verbosity 1
Debug: Remote helper: Waiting...
Debug: Remote helper: <- unsupported
Debug: Remote helper: Waiting...
Debug: Remote helper: <- @refs/heads/master HEAD
Debug: Remote helper: Waiting...
Debug: Remote helper: <- ? refs/heads/master
Debug: Remote helper: Waiting...
Debug: Remote helper: <-
Debug: Read ref listing.
Debug: Remote helper: -> export
Debug: Remote helper: Waiting...
ERROR: unhandled export command: mark :4
Debug: Remote helper quit.
error: failed to push some refs to 'hg::../test.hg'
Debug: Disconnecting.

[fingolfin]

Huh, that's weird, why does it have "e309a3ed3ccfb63a4e5c2b3f6edf391a1dbef14f" as remote name in there? This looks more like what I'd expect from gitifyhg...

[fingolfin]

Please also tell me the output of these two commands:

git config remote-hg.hg-git-compat
git config remote-hg.track-branches

[alexei-31]

C:\Temp\test\test.git>set GIT_TRANSPORT_HELPER_DEBUG=1

C:\Temp\test\test.git>git config remote-hg.hg-git-compat

C:\Temp\test\test.git>git config remote-hg.track-branches
false

[fingolfin]

Hmm, I don't see how, but I notice that you have remote-hg.track-branches set to false. Is that intentional? Could you perhaps set it to true temporarily, and re-run the test, to make sure it is not causing the problems? (Conversely, I just tried your test with it set to false, but still could not reproduce the issue)

[fingolfin]

Also, can you give me a list of files in the .git/hg directory?

[alexei-31]

Result of work command dir .git\hg /S in folder C:\Temp\test\test.git:

 Folder C:\Temp\test\test.git\.git\hg

04.12.2015  22:14    <DIR>          .
04.12.2015  22:14    <DIR>          ..
04.12.2015  22:14    <DIR>          origin
               0 files              0 bytes

 Folder C:\Temp\test\test.git\.git\hg\origin

04.12.2015  22:14    <DIR>          .
04.12.2015  22:14    <DIR>          ..
04.12.2015  22:14               220 marks-git
04.12.2015  22:14               225 marks-hg
               2 files            445 bytes

[alexei-31]

Content of marks-hg (I added the newlines to eliminate scrolling):

{"tips": {"bookmarks/master": "a2ae2410a663691a16901c11d3ffbb9cc2e802b3"}, 
"last-mark": 5, "last-note": 5, "version": 2, 
"marks": {"a9f7eeb5ee812829b87d8065a6aaa2155430cfab": 2,
"a2ae2410a663691a16901c11d3ffbb9cc2e802b3": 4}}