Use single git repo for multiple notebooks

Like the title says - I'd like to have a single .git dir in ~/.nb instead of multiple .git directories for each notebook. This would make syncing across multiple machines much easier.

This would be a nice feature to have. This could be either:

1. Each notebook as a directory of the git repository or,
2. Each notebook as a branch of the git repository.

I also have this problem, and I tend to solve it through settings. By default, each notebook has a separate. Git, but it can be adjusted to build. Git for the entire project. In this way, the implementation will be more flexible and easy to use. The reason why I am here is that today, because of this problem, my whole project has caused a version conflict, and I have to solve this problem manually

我也遇到了这个问题，我倾向于通过设置来解决这个问题。 默认还是每个笔记本单独一个.git，但是可以调整为对整个项目构建.git。 这样的话，在实现上会更加的灵活好用。 我之所以在这里是因为今天因为这个问题导致了我的整个项目引起了版本冲突，我不得不手动去解决这个问题

Just want to say I’m thinking about this and am still searching for a clear vision about how to implement it cleanly. I have some vague ideas about how to implement it using a plugin, and I can sort of envision how we could hijack the notebook list in the header to display the top level folders in the current notebook. There are a lot of little UX things that will need to be sorted out, like completions. I’m currently building out the folder functionality and there are some other things I want to do that should give us a foundation for this.

Started using nb recently and love it! Would also be interested in this feature. Have you thought about using git modules in the .nb directory? So when a new remote is added for a notebook, implementing a hook to add that remote as a submodule to the .gitmodules file?

@kyphae Glad you like it, and thanks for the suggestion! I’m doing some experiments locally and I think I’m zeroing in on an implementation.

Hi,

great Job, I like the app.

1. was using VimWiki. so it seems to be so that it integrates well with this VimWiki Plugin. May be some more integartion from Vim will be nice, fro seacrhing etc ...

2. I laos like to have a signle .git repo for all the Notebooks I tried to realize it with submodule or suboldder, but di not get any sucess.

so if someone has an idea, I ll paeciate to share it.

Thanks

Multiple notebooks can now be synced with a single remote using orphan branches.

nb remote set now includes additional prompts enabling the user to create a new orphan branch when adding a remote. nb init, nb notebooks add, and nb notebooks init now accept a final <branch> option that can be used to clone a specific branch.

The orphan branch approach is used here because the orphan branch feature appears to be stable in git and this implementation simply extends the existing remote functionality in nb. Conceptually, I think it makes a lot of sense and appears to be the most logical way to store multiple unrelated histories in one repository. There is test coverage:

bats test/sync* test/remote* test/init.bats test/notebooks-add.bats test/notebooks-init.bats

Information in the README:

https://github.com/xwmx/nb#syncing-multiple-notebooks-with-one-remote

This is absolutely beautiful and works perfectly. There is only one minor obstacle I've encountered: all of the remote objects will be copied to the notebook's local repository, including the objects from unrelated branches.

This means that if there is an unrelated branch with an enormous file in it, the notebook's local repository will reflect that size, even if its orphan branch has no knowledge of the file. This will quickly become problematic when having multiple notebooks point to a repository with an unrelated but large branch.

However, that's a Git thing, rather than a problem with nb. I wonder if nb could magically work around that. Knowing Git's quirks, this would be quite the rabbit hole to fall into. The current solution is brilliant as it is: the use of orphan branches is elegance at its finest.

---

Update: it looks like there is a way for nb to robustly avoid the cloning/fetching of unrelated objects:

Action	Command
Cloning	git clone --no-local --single-branch --branch <orphan branch name> <repo url> <notebook name>
Fetching	git fetch <remote name> <orphan branch name>

Note: --no-local is optional when it comes to non-local cloning.

@yumiris Thanks for the kind words, and thank you so much for figuring out the solution and providing the exact commands. All of the cloning, fetching, and branch handling has been updated as described so now only the current branch for each notebook is tracked locally. These changes are available in the current version, 6.1.5.

My pleasure! It's all working overall very well, with only a quirk which I've managed to reproduce. Don't hesitate to let me know if you need more information.

When setting an existing notebook's remote to an existing remote orphan branch, Git reports the following problem:

fatal: ambiguous argument '<orphan branch name>': unknown revision or path not in the working tree.

Steps to reproduce

Remotely We'll create an empty bare repository for simplicity's sake:

git init --bare repository

Locally

1. Initialise two local notebooks (x and y in this example).
2. Set notebook x's remote to <remote url> shared-branch orphan. It'll work just fine.
3. Set notebook y's remote to <remote url> shared-branch orphan. Here's where it all breaks.

nb notebooks init ./x
# Initialized local notebook: ./x

nb notebooks init ./y
# Initialized local notebook: ./y

cd x
nb remote set git@server:repository shared-branch

# Adding remote to: local
# -----------------------
# URL:    git@server:repository
# Branch: shared-branch
# --------------
# Proceed? [y/N] y

# Remote set to: git@server:repository (shared-branch)

cd y
nb remote set git@server:repository shared-branch

# Adding remote to: local
# -----------------------
# URL:    git@server:repository
# Branch: shared-branch
# --------------
# Proceed? [y/N] y

# fatal: ambiguous argument 'shared-branch': unknown revision or path not in the working tree.
# Use '--' to separate paths from revisions, like this:
# 'git <command> [<revision>...] -- [<file>...]'

Miscellaneous

Setting a separate branch name works fine:

nb remote set git@repository:repository another-branch

# Adding remote to: local
# -----------------------
# URL:    git@server:repository
# Branch: another-branch
# --------------
# Proceed? [y/N] y
# Branch not present on remote: another-branch

# [1] Merge and sync with an existing remote branch.
# [2] Sync as a new orphan branch on the remote.

# Choose an option or q to quit: 2
# Remote set to: git@server:repository (another-branch)

Workaround

git checkout -b notebook-name
git remote set-url origin git@server:repository
git fetch origin notebook-name
git branch -u origin/notebook-name notebook-name
nb sync

@yumiris Awesome. I really appreciate your help and the detailed information. I was able to take those operations, turn them directly into a couple of test cases, and identify and resolve the issue. I also went deeper into this feature and made various additional improvements to the behavior, prompts, and tests, which are now available in version 6.2.0.

There was a small bug so I took the opportunity to squeeze in some additional functionality in the patch, which is available as of version 6.2.2.

Notably, as far as I understand, git does not provide the ability to delete or rename the primary branch for a remote. In order to work around this I’ve added the ability to reset any remote branch to a blank state, and this functionality is used in place of delete when using remote remove with a notebook that’s syncing with a remote’s primary branch.

At this point the branch handling should be working pretty nicely. There is pretty good test coverage here. Let me know if there are any issues or gaps.

nb help remote

Absolutely beautiful stuff-- it takes a deity to deal with Git's intricacies and oddities, especially when Bash is the only weapon being used against the mighty beast.

Will let you know if anything pops up, and many thanks again for the immediate and thorough update. I hope it hasn't been too much of a pain!

Been playing with nb since last year and dove right into using all the new features that ensued from this issue, around orphan branches for multiple notebooks. Awesome! I have a lot of computers and have what might be a feature request, or is it existing functionality I haven't discovered yet? In any case, it seems most closely related to this issue so posting here to see where this goes..

@xwmx if my nb repo on GitHub has n orphan branches and I want to add n notebooks for each of those on a new computer, is there a way to nb notebooks add all n notebooks in one go rather than doing this one by one? The repo itself does not have the folder structure that nb has, so I understand nb has to create the folder structure inside NB_DIR for each notebook each with its own copy of the repo set to the corresponding orphan branch. But for a repo with a lot of notebooks, times a lot of computers on which to setup my nb, it's pretty hard to do this for large n. At least compared to, say, a simple git clone operation which pulls all of the notebooks branches in a single command. I even tried just setting NB_DIR to the cloned repo to see if I'd get all my notebooks automatically that way but in hindsight that obviously doesn't work.

Would an nb n add --all <remote> command be feasible, and then it builds out all notebooks in NB_DIR from all orphan branches found?

And / Or, an nb n add <remote> <list of branches> so that you can basically "select" the notebooks you want to add from the repo in a single command, not necessarily all?

Maybe the answer is that I just have to put the add in a loop and iterate over the branches I want and do this myself.

@aguevara Indeed. Thanks for reminder. Both of the approaches you mention sound interesting. I intend to explore this more when I can work out the details.

@aguevara notebooks add has been updated so it behaves like your recommendations. These changes are available in 6.11.1.

nb notebooks add <url> [<branch>... | --all]

When there are multiple branches, a prompt is displayed for selecting a branch or "All Branches". Notebook names are derived from the branch names, and the user is prompted to accept each name or provide a different one.

Branches can be selected by passing one or more branch names to notebooks add, or --all for all branches.

Works like a charm, thanks so much @xwmx. This new feature is a huge time saver!