git-filter-repo

git-filter-repo copied to clipboard

I'm trying to merge two repositories together, while keeping a nice history. I nearly have a working solution, but I'm running into a problem, which could be solved by an extension to git-filter-repo.

Current process

In this example, repo A is the donor repo and repo B is the recipient repo.

1. Use git-filter-repo with --path-rename to rename repo A to put all contents in a subdirectory
2. Add repo A as a remote of repo B.
3. Pull repo A from repo B
4. Take the initial commit from repo A, and use git replace --graft to graft the commit into the current tip of repo B.
5. Use git-filter-repo to make the graft into a real commit which can be pushed and pulled.

Problem with current process

If you try to check out any of the commits from repo A, you find that none of the contents of repo B are present at those commits. This is because the first commit from repo A "deletes" the contents outside of the subdirectory.

Review of other solutions

• I could use rebase to stack these commits atop each other. However, I want to preserve the structure of the repository, including any merges.
• I could use git rebase -i --rebase-merges. This allows me to preserve the merges, but it requires that I re-resolve any merge conflicts. This is unsatisfying, because some of the conflicts are hairy and require non-trivial choices about what to keep. I will probably introduce bugs doing this. You can kind of avoid this with git rerere, but it doesn't work very reliably.
• I could merge the unrelated histories. However, this results in a repository with multiple root commits.

Feature request

Essentially, I'm looking for a solution which will let me take two unrelated series of commits, and stack one on top of the other. I'm not looking for it to resolve conflicts between those two series - the two sets of commits modify different files.

Did you try that, instead of grafting repo A to the tip of repo B, do a graft to the first commit in repo B and then merge the repo A branch to the tip of repo B?

I'm facing exactly the same issue and I would love to know how to overcome this.

Let's say I have the following tree structure:

     /-E
B-C-D
    \ -F

...and would like to add an extra commit at the beginning with a new file (e.g. README.md):

         /-E
A-B-C-D
        \ -F

Unfortunately git replace --graft B A will cause B to being appended with the removal of the new files in A.

For now I'm rebasing all the branches in the following way:

git branch --contains B  | grep -v HEAD | xargs -I % git rebase --rebase-merges --root --onto A %

Problem with current process

If you try to check out any of the commits from repo A, you find that none of the contents of repo B are present at those commits. This is because the first commit from repo A "deletes" the contents outside of the subdirectory.

Right, when you create the graft, it just lets the root of A have the tip of B as a parent, but leaves the root of A with the same tree, meaning it looks like it deleted all the files from the tip of B.

Feature request

Essentially, I'm looking for a solution which will let me take two unrelated series of commits, and stack one on top of the other. I'm not looking for it to resolve conflicts between those two series - the two sets of commits modify different files.

Since the original root of A had no file deletions of its own (as a root commit, it can only add files), that means that all the deletions now shown when you inspect it in git log or git fast-export come from it being a graft. Since you don't want any of those deletions, that suggests just removing them will fix the problem. First, though, the hard part -- identifying the replacement commit. When you ran e.g.

$ git replace --graft $ROOT_OF_A $TIP_OF_B

then git created a replacement reference. If you only have one, then this will be easy; just run

$ git show-ref | grep replace

which will give output like

fa79395094341a9bb3a6052bd1f70ebb663801af refs/replace/fac4821d155fb47e056d0e89bf56a32039c1cf80

If you have more than one line of output, you'll need to feed the hashes on the right hand side to git log to see which one is relevant. In this case, I'll assume you only have one and use fac4821d155fb47e056d0e89bf56a32039c1cf80 in my example. So here, I'd run:

git filter-repo --commit-callback 'if commit.original_id == b"fac4821d155fb47e056d0e89bf56a32039c1cf80": commit.file_changes = [x for x in commit.file_changes if x.type != b"D"]'

That will modify that commit to not delete all those files (i.e. leaving all the files from the history of B intact for the first commit of A), and since other commits in A didn't delete or modify those files, they'll stick around.

Make sense?

I'm facing exactly the same issue and I would love to know how to overcome this.

Let's say I have the following tree structure:

     /-E
B-C-D
    \ -F

...and would like to add an extra commit at the beginning with a new file (e.g. README.md):

         /-E
A-B-C-D
        \ -F

You could solve it similarly as I mentioned above for @nickodell 's usecase, if you continue using replace refs.

Alternatively, I'll note that there is a contrib/filter-repo-demos/insert-beginning script that solves a very similar case. Instead of adding a new root commit, though, it simply modifies the existing root commit(s) to include the additional file. So it's not quite the problem you were describing, but it might have useful code or inspiration for your problem.

Interesting, I'll have to try that out. I'll report back if I get around to it.

On Tue, Nov 9, 2021, 6:26 PM Elijah Newren @.***> wrote:

I'm facing exactly the same issue and I would love to know how to overcome this.

Let's say I have the following tree structure:

 /-E

B-C-D \ -F

...and would like to add an extra commit at the beginning with a new file (e.g. README.md):

     /-E

A-B-C-D \ -F

You could solve it similarly as I mentioned above for @nickodell https://github.com/nickodell 's usecase, if you continue using replace refs.

Alternatively, I'll note that there is a contrib/filter-repo-demos/insert-beginning script that solves a very similar case. Instead of adding a new root commit, though, it simply modifies the existing root commit(s) to include the additional file. So it's not quite the problem you were describing, but it might have useful code or inspiration for your problem.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/newren/git-filter-repo/issues/170#issuecomment-964696251, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFCWIV6IMK3FRG5CEODXOLULHC5FANCNFSM4TFQ4CFQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Interesting, I'll have to try that out. I'll report back if I get around to it.

I'll assume you didn't get around to it...or that it worked perfectly but you got busy and didn't get a chance to report back. Feel free to reopen if my assumptions are bad.

I'm going to level with you, I don't remember opening this issue.

I think I solved this via merging unrelated histories, and just lived with the multiple root commits/unrelated history.