gitbutler icon indicating copy to clipboard operation
gitbutler copied to clipboard

Support clean / smudge filtering

Open krlvi opened this issue 1 year ago • 8 comments

krlvi avatar Feb 21 '24 18:02 krlvi

libgit has it https://github.com/libgit2/libgit2/issues/1473

krlvi avatar Feb 21 '24 18:02 krlvi

The libgit2 site is down for me at the moment, here is the wayback from last week of the API docs: https://web.archive.org/web/20240212173946/https://libgit2.org/libgit2/#HEAD/group/filter

droidicus avatar Feb 21 '24 18:02 droidicus

Yeah, I broke the DNS for libgit2.org. 😓 It should be back now: https://libgit2.org/libgit2/#HEAD/group/filter

We recently taught libgit2 to invoke external processes - like ssh. The other use for this is, obviously, external filter drivers (like LFS). I have a branch somewhere that I can push up if this would be helpful for us to iterate on together.

ethomson avatar Feb 22 '24 10:02 ethomson

Yeah, I think the libgit2 filter functions aren't implemented in git2-rs yet. It's probably not super common to have much other than crlf and lfs filters, so we could special case those, OR, we could fully implement smudge/clean stuff with custom commands. [heavy sigh]. I'm not sure how much work that is. Maybe we could start with the simple, common stuff and see what that looks like first.

schacon avatar Feb 22 '24 16:02 schacon

Do you want to have your own filters that you customize? Or do you just want to support filter.*.smudge and filter.*.clean and have them Just Work?

If it's the former, yeah, that's hard and requires git2-rs. If it's the latter, let's let libgit2 just Run The Things For You.

I imagine some sort of setting like git_libgit2_set_opt(GO_RUN_FILTERS_FOR_ME, 1) and then voila.

I'll see if I can dig up my branch — most of the hard bits were actually here: https://github.com/libgit2/libgit2/pull/6617

ethomson avatar Feb 22 '24 16:02 ethomson

@ethomson I had originally talked to @krlvi on discord about this. My original question was:

I use both Python and iPython Notebooks in my projects, and use https://pypi.org/project/nbstripout/0.2.5/ installed as a filter in my repos to prevent the runtime changes from affecting the notebook files. However, it appears that the changes typically filtered by git are showing up in git-butler, and this is causing failures in branches where I have changed *.ipynb files. Does git-butler currently support git clean/smudge filters, or is this broken in some other way?

I don't have any need to create new filters, I just need to run the filters that already exist in the repository with git on the CLI. Currently this configuration results in broken branches, so I cannot use git-butler for my use case.

droidicus avatar Feb 23 '24 01:02 droidicus

I guess the lack of support for smudge filters is the reason why all the files encrypted with git-crypt are suddenly listed in a branch? This only started happening today though, after I updated to 0.12.2, and it creates a mess with several hundred files obscuring any real changes. So far the only workaround I've come up with is to just move all these files to a separate branch and keep another branch as the default, but they're still an eyesore, and I need to keep adding a new default branch if I delete the second last one.

tfylling avatar Jun 07 '24 14:06 tfylling