How to merge .ipynb with git?

[iamtodor]

Hello, guys! I installed this tool for merging notebooks. I have a conflict in a file test.ipynb. I was trying the following nbmerge test.ipynb, but it gives me an error. I have read your page https://nbdime.readthedocs.io/en/latest/merging.html# but I can't figure out how to merge in a way that was provided on the screen https://nbdime.readthedocs.io/en/latest/_images/nbmerge-web.png Could you please illuminate how to merge a single file in the browser or in some different way?

[vidartf]

Hi @iamtodor ! There are a few things that would be helpful for context in order to troubleshoot this:

• Are you using git?
• Did you install nbdime as a merge driver and or merge tool?
• Which version of nbdime are you using?
• Are there any other git extensions installed (e.g. git filters)?
• What are the errors messages you are seeing?
• Could you post a full list of steps needed to reproduce the error? (this should help answer several of the questions above)

In general, merging can be done in two ways:

1. Opening a conflicted notebook and resolving anything indicated by merge markers (>>>>>>> etc).
   • If you are using git, install the nbdime merge driver.
   • If you are not using git, call nbmerge [base.ipynb] local.ipynb remote.ipynb --out output.ipynb.
   • Open the conflicted notebook (or output.ipynb) as a normal notebook and edit it.
   • If using git, mark the conflict as resolved by using git add/git commit as for other files.
2. Using nbdime's merge tool.
   • If using git, use the merge tool integration by calling nbdime mergetool once a conflict is identified.
   • If not using git, call nbmerge-web [base.ipynb] local.ipynb remote.ipynb --out output.ipynb.

[iamtodor]

• Yes, I am using git.
• Not sure if I understand correct "merge drive and or merge tool". I installed it as pip install nbdime
• Nope
• The error: For the command nbdime mergetool --tool=nbdime assignments/assignment2/PyTorch.ipynb I see the following error:

(env) mcbk:assignment2 todor$ nbdime mergetool --tool=nbdime assignments/assignment2/PyTorch.ipynb
Skipping given filenames without .ipynb extension: ['--tool=nbdime']
No files need merging

For the command nbdime mergetool I see the following:

(env) mcbk:assignment2 todor$ nbdime mergetool assignments/assignment2/PyTorch.ipynb
No files need merging

And the last one is strange because after pulling I have seen

Auto-merging assignments/assignment2/PyTorch.ipynb
CONFLICT (content): Merge conflict in assignments/assignment2/PyTorch.ipynb
error: Failed to merge in the changes.
Patch failed at 0015 implemented part of pytorch

And git status shows me

(env) mcbk:assignment2 todor$ git status
rebase in progress; onto 155333e
You are currently rebasing branch 'master' on '155333e'.
  (fix conflicts and then run "git rebase --continue")
  (use "git rebase --skip" to skip this patch)
  (use "git rebase --abort" to check out the original branch)

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

	modified:   ../../README.md
	modified:   gradient_check.py
	modified:   layers.py
	modified:   model.py

Unmerged paths:
  (use "git reset HEAD <file>..." to unstage)
  (use "git add <file>..." to mark resolution)

	both modified:   PyTorch.ipynb

(env) mcbk:assignment2 todor$

[iamtodor]

I was trying the following seems it helped for me a bit:

(env) mcbk:dlcourse_ai todor$ nbdime mergetool assignments/assignment2/PyTorch.ipynb
Merging:
assignments/assignment2/PyTorch.ipynb

Normal merge conflict for 'assignments/assignment2/PyTorch.ipynb':
  {local}: modified file
  {remote}: modified file
Unknown merge tool nbdime

In my directory appear a few new files:

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	assignments/assignment2/PyTorch_BACKUP_20246.ipynb
	assignments/assignment2/PyTorch_BASE_20246.ipynb
	assignments/assignment2/PyTorch_LOCAL_20246.ipynb
	assignments/assignment2/PyTorch_REMOTE_20246.ipynb

But I still dont undestand how to merge like you provided on the screenshot https://nbdime.readthedocs.io/en/latest/_images/nbmerge-web.png :(

[translunar]

I think I'm having similar problems. Even after setting up nbdime, I have to force it to use nbdime as the mergetool: git mergetool --tool=nbdime.

When it opens the browser tab, the three columns aren't rendered in a row; instead, the columns are displayed vertically. Moreover, there don't appear to be any controls or a method for telling it which pieces to keep.

[iamtodor]

Thank you I am not alone. Thought maybe I am dumbass who can't use regular tool as it was intended. The thing I wish to have is explicit example from A to Z how to merge conflicted notebook when pulling via got

[vidartf]

To use nbdime with git, you need to "register" it (configure git so it knows about nbdime).

The shortest way to configuring this is to follow the quick start guide: https://nbdime.readthedocs.io/en/latest/index.html#git-integration-quickstart :

nbdime config-git --enable --global

The thing I wish to have is explicit example from A to Z how to merge conflicted notebook when pulling via got

That would be a very nice resolution of this issue once we've got you settled. If you would want to contribute by writing a draft for such a guide (either as a PR or a comment here), that would really help me out!

[iamtodor]

Thank you, nbdime config-git --enable --global helped. I think the command nbdime config-git --enable --global is the key, and after that run git mergetool --tool=nbdime

[translunar]

I had used nbdime config-git --enable without the --global. Should this not be sufficient?

[iamtodor]

Even with the registry, I couldn't resolve the conflicts. The picture similar to https://nbdime.readthedocs.io/en/latest/_images/nbmerge-web.png has appeared, but not like that: not 3 separated columns. And I didn't find any button to apply my/remote changes or merge line by line.

Here how this merge process looks like to me: http://i.imgur.com/hb7B4cn.png

[iamtodor]

Somehow I can't resolve conflicts by my own according to this output:

Falling back to patching base and 3-way merge...                                                                                                                          
[W strategies:469] Recording unresolved conflicts in /metadata/nbdime-conflicts.                                                                                          
Traceback (most recent call last):                                                                                                                                        
  File "/home/ilyat/projects/course/dlcourse_ai/assignments/assignment1/env/bin/git-nbmergedriver", line 10, in <module>                                                  
    sys.exit(main())                                                                                                                                                      
  File "/home/ilyat/projects/course/dlcourse_ai/assignments/assignment1/env/lib/python3.6/site-packages/nbdime/vcs/git/mergedriver.py", line 115, in main                 
    return nbmergeapp.main_merge(opts)                                                                                                                                    
  File "/home/ilyat/projects/course/dlcourse_ai/assignments/assignment1/env/lib/python3.6/site-packages/nbdime/nbmergeapp.py", line 51, in main_merge
    merged, decisions = merge_notebooks(b, l, r, args)
  File "/home/ilyat/projects/course/dlcourse_ai/assignments/assignment1/env/lib/python3.6/site-packages/nbdime/merging/notebooks.py", line 184, in merge_notebooks
    merged = apply_decisions(base, decisions)
  File "/home/ilyat/projects/course/dlcourse_ai/assignments/assignment1/env/lib/python3.6/site-packages/nbdime/merging/decisions.py", line 649, in apply_decisions
    parent[last_key] = patch(resolved, diffs)
  File "/home/ilyat/projects/course/dlcourse_ai/assignments/assignment1/env/lib/python3.6/site-packages/nbdime/patching.py", line 133, in patch
    return patch_dict(obj, diff)
  File "/home/ilyat/projects/course/dlcourse_ai/assignments/assignment1/env/lib/python3.6/site-packages/nbdime/patching.py", line 96, in patch_dict
    assert key not in newobj, 'multiple diff entries target same key: %r' % key
AssertionError: multiple diff entries target same key: 'nbdime-conflicts'
Auto-merging assignments/assignment2/PyTorch.ipynb
CONFLICT (content): Merge conflict in assignments/assignment2/PyTorch.ipynb
error: Failed to merge in the changes.
Patch failed at 0022 update
Use 'git am --show-current-patch' to see the failed patch

Despite on file looks good for me, no conflicts were detected and it loads well with jupyter lab

[vidartf]

@mohawkjohn nbdime config-git --enable will only enable nbdime for the current repository, i.e. the repository folder you are in when you run the command. That might or might not be what you want. Hopefully that makes it clearer.

[vidartf]

@iamtodor That looks like a bug with the mergetool. Would you mind sharing the notebooks in question (maybe just the relevant cells if you have something else you want to keep private)? Once I can reproduce it locally, I could try to fix it 😃

[translunar]

Whether global or local, I'm having the same issue as OP. I cannot share the notebooks with you, unfortunately. Here's an example image:

[TommasoBendinelli]

I get the same issue, I do not know how to merge because the merging tool appears not to work properly

[vidartf]

While I will have to look into the merge tool in detail to figure out what is going wrong there, for now I just want to make it clear that you can still do merges in the following way:

• When there is a merge conflict, open the conflicted notebook as you normally would (in classic notebook, or jupyterlab).
• The notebook should contain conflict markers >>>>>>>, <<<<<<< etc.
• Edit the notebook so it has the final state you want (resolving the marked areas).
• When the conflicts are resolved, call git add <file path> as normal to mark the conflicts as having been resolved.
• When all files are resolved, continue with git as normal (git commit / git rebase --continue / git merge --continue, etc).

[iamtodor]

@vidartf thank you for the attention to the problem. Unfortunately, as you can see a lot of people are facing this issue and I don't think this problem is caused by the tool itself but probably because of the lack of documentation and examples. It would be highly appreciated if you provide us some onboarding (tutorial) step by step how to use your amazing tool :)