🐛 git add -p one-to-one correspondence errors

[Minnozz]

When I try to use git add -p, I get the following error:

$ git add -p
fatal: mismatched output from interactive.diffFilter
hint: Your filter must maintain a one-to-one correspondence
hint: between its input and output lines.

I'm using delta with the following config:

$ delta --version
delta 0.4.3
$ delta --show-config
    commit-style                  = raw
    file-style                    = blue
    hunk-header-style             = syntax
    minus-style                   = syntax 52
    minus-non-emph-style          = syntax 52
    minus-emph-style              = syntax 88
    minus-empty-line-marker-style = normal 88
    zero-style                    = syntax
    plus-style                    = syntax black
    plus-non-emph-style           = syntax black
    plus-emph-style               = syntax 22
    plus-empty-line-marker-style  = normal 22
    whitespace-error-style        = reverse magenta
    24-bit-color                  = false
    file-added-label              = 'added:'
    file-modified-label           = ''
    file-removed-label            = 'removed:'
    file-renamed-label            = 'renamed:'
    hyperlinks                    = false
    inspect-raw-lines             = true
    keep-plus-minus-markers       = true
    line-numbers                  = true
    line-numbers-minus-style      = red
    line-numbers-zero-style       = brightgreen
    line-numbers-plus-style       = green
    line-numbers-left-style       = blue
    line-numbers-right-style      = blue
    line-numbers-left-format      = '{nm:^4}⋮'
    line-numbers-right-format     = '{np:^4}│'
    max-line-distance             = 0.6
    max-line-length               = 512
    navigate                      = false
    paging                        = auto
    side-by-side                  = false
    syntax-theme                  = Solarized (dark)
    width                         = 145
    tabs                          = 4
    word-diff-regex               = '\w+'

In my .gitconfig are the following relevant options:

[core]
        pager = delta
[delta]
        commit-style = raw
        file-style = blue
        hunk-header-style = syntax
        minus-style = syntax 52
        minus-emph-style = syntax 88
        zero-style = syntax normal
        plus-style = syntax black
        plus-emph-style = syntax 22
        keep-plus-minus-markers = true
        line-numbers = true
        line-numbers-minus-style = red
        line-numbers-zero-style = brightgreen
        line-numbers-plus-style = green
        whitespace-error-style = reverse magenta
        minus-empty-line-marker-style = normal 88
        plus-empty-line-marker-style = normal 22
[interactive]
        diffFilter = delta --color-only
[color]
        ui = true

[dandavison]

Hi @Minnozz, thanks for this, very helpful. Clearly the test suite needs to be strengthened for git add -p also. I know git add -p is important, so this is probably the top priority for delta bugs right now.

[Minnozz]

Hi @dandavison, let me know if you need more information.

[ghost]

I looked for causes of this bug before, and

side-by-side = true
hunk-header-style = omit (only omit. I usually use this option except when git add -p)
file-style = (any option interestingly)

these options made it happened. @Minnozz 's config can work if you remove file-style one.

[Kr1ss-XD]

I'd like to note that git add -p works here although I'm using file-style. Maybe it does not play well in combination with some other option(s) ?

For reference, my delta --show-config.

EDIT Oddly enough this config doesn't break add -p either in my environment. It contains all of the options mentioned above by @ryuta69.

[ghost]

Thank you @Kr1ss-XD for looking. Hmm, I tried same config with you, however, it doesn't work. (core.pager = delta, and interactive.diffFilter = delta --color-only)

As picture showing,

❯ git --version
git version 2.28.0 (latest)
❯ delta --version
delta 0.4.3 (latest)

OS is MacOS Catalina 10.15.6.

Additionally, with your config, if I delete

file-style                    = bold 232 bright-yellow
hunk-header-style             = omit normal
side-by-side                  = true

now it works.

[Kr1ss-XD]

That's weird.

I'm trying @Minnozz 's configuration as I'm writing. It seems if the file option is deleted, it actually works, just as you suggested.

FWIW,

$ git --version; delta --version; uname -smro
git version 2.28.0
delta 0.4.3
Linux 5.7.19-1-ck x86_64 GNU/Linux

I hope this helps chasing that bug.

[Kr1ss-XD]

I think I found something :

If I put the file-style option in a feature category, git add -p works with the @Minnozz configuration !

[delta]
    features = files
    <... other options ...>
[delta "files"]
    file-style = blue

cc @dandavison

EDIT The same applies to hunk-header-style = omit.

[ghost]

@Kr1ss-XD Oh! Same as hunk-header-style, and side-by-side. Moving them to features, git add -p works while usual git diff work as those options enabled!

I think

src/options/set.rs
    if config::user_supplied_option("color-only", arg_matches) {
        builtin_features.remove("side-by-side");
    }

this is the reason why side-by-side works. I look for file-style, hunk-header-style's reasons.

[dandavison]

Thanks a lot @Kr1ss-XD and @ryuta69 for investigating here.

I think @Kr1ss-XD's suggestion of using features to work around these issues is an excellent one for now, until we fix all the git app -p bugs.

@Kr1ss-XD I'm finding that I have to move both the file-style and hunk-header-style to a feature to make @Minnozz's config work under git add -p.

So, to be completely explicit, here is a modification of @Minnozz's config that I am finding works.

[delta]
    features = meta
    commit-style = raw
    # file-style = blue
    # hunk-header-style = syntax
    minus-style = syntax 52
    minus-emph-style = syntax 88
    zero-style = syntax normal
    plus-style = syntax black
    plus-emph-style = syntax 22
    keep-plus-minus-markers = true
    line-numbers = true
    line-numbers-minus-style = red
    line-numbers-zero-style = brightgreen
    line-numbers-plus-style = green
    whitespace-error-style = reverse magenta
    minus-empty-line-marker-style = normal 88
    plus-empty-line-marker-style = normal 22

[delta "meta"]
    file-style = blue
    hunk-header-style = syntax

[delta "diff-filter"]
    # Can leave empty, but optionally put diffFilter-only settings here

[interactive]
    # I've added a --features override here to be explicit that we don't want diffFilter to use the
    # features defined in the main [delta] section.
    diffFilter = delta --color-only --features=diff-filter

[dandavison]

I look for file-style, hunk-header-style's reasons.

@ryuta69 It would be awesome if you can make progress here! I'm not certain but I think https://github.com/dandavison/delta/pull/272#discussion_r474371089 may be relevant.

[ghost]

[WIP]

commit_style = omit
hunk-header-decoration-style = {any option}

also triggers this bug. (It's because it removes commit hash.)

I build test, then I realized only commit-style=omit, file-style={any except raw}, and hunk-header-style={any except raw} cause this bug.

It's still WIP, but gonna send PR in few days.

[kevinhwang91]

[core]
    pager = delta

[delta]
    file-decoration-style = ul

[interactive]
    diffFilter = delta --color-only

also produce this issue. I must use this workaround replaces diffFilter = delta --color-only with diffFilter = delta --color-only --file-decoration-style=omit.

[dandavison]

@kevinhwang91 could you confirm your delta version please?

[kevinhwang91]

@kevinhwang91 could you confirm your delta version please?

delta --version delta 0.4.4

[ghost]

Sorry, I missed it. I'm going to add fix, and check other options again.

[dandavison]

Remaining bugs here should be fixed in master thanks to @ulwlu's PR #367. I'll post when it's released.

[imiric]

I'm having the same issue on Linux and v0.7.1, and confirmed that the suggested features workaround works.

FWIW here's my config:

$ delta --version
delta 0.7.1
$ delta --show-config
    commit-style                  = raw
    file-style                    = bold normal
    hunk-header-style             = normal
    minus-style                   = syntax "#340001"
    minus-non-emph-style          = syntax "#340001"
    minus-emph-style              = normal "#901011"
    minus-empty-line-marker-style = normal "#3f0001"
    zero-style                    = syntax
    plus-style                    = syntax "#033500"
    plus-non-emph-style           = syntax "#033500"
    plus-emph-style               = syntax "#006000"
    plus-empty-line-marker-style  = normal "#002800"
    whitespace-error-style        = reverse red
    24-bit-color                  = true
    file-added-label              = 'added:'
    file-modified-label           = Δ
    file-removed-label            = 'removed:'
    file-renamed-label            = 'renamed:'
    hyperlinks                    = false
    inspect-raw-lines             = true
    keep-plus-minus-markers       = false
    line-numbers                  = true
    line-numbers-minus-style      = 88
    line-numbers-zero-style       = "#444444"
    line-numbers-plus-style       = 28
    line-numbers-left-style       = blue
    line-numbers-right-style      = blue
    line-numbers-left-format      = '{nm:^4}⋮'
    line-numbers-right-format     = '{np:^4}│'
    max-line-distance             = 0.6
    max-line-length               = 512
    navigate                      = true
    navigate-regexp               = ^(commit|Δ|added:|removed:|renamed:)
    paging                        = auto
    side-by-side                  = false
    syntax-theme                  = Monokai Extended
    width                         = 190
    tabs                          = 4
    word-diff-regex               = '\w+'

And the relevant parts of .gitconfig:

[color]
    ui          = auto
[core]
    pager       = delta
[delta]
    features = files
    line-numbers = true
    plus-style = "syntax #033500"
    minus-style = "syntax #340001"
    syntax-theme = Monokai Extended
    hunk-header-style = "file line-number"
    hunk-header-decoration-style = "ul ol"
    navigate = true
[delta "files"]
    file-style = "box bold"
[interactive]
    diffFilter = delta --color-only

[lithammer]

Seems like diff-so-fancy has managed to add support git add -p while keeping the formatting, see https://github.com/so-fancy/diff-so-fancy/pull/390.

[dandavison]

Seems like diff-so-fancy has managed to add support git add -p while keeping the formatting, see so-fancy/diff-so-fancy#390.

@lithammer Yes, and that is what Delta does, since 0.4.5 (Dec 2020). You can't use side-by-side of course, since there is then no correspondence with the true git patch, but otherwise delta can be used with git add -p (% the bug that @imiric is pointing out):

[interactive]
    diffFilter = delta --color-only

@imiric Thanks, I can reproduce your bug -- using box with file-style causes the one-to-one-correspondence error (unless you use that strange features workaround).

[lithammer]

Seems like diff-so-fancy has managed to add support git add -p while keeping the formatting, see so-fancy/diff-so-fancy#390.

@lithammer Yes, and that is what Delta does, since 0.4.5 (Dec 2020). You can't use side-by-side of course, since there is then no correspondence with the true git patch, but otherwise delta can be used with git add -p (% the bug that @imiric is pointing out):

[interactive]
    diffFilter = delta --color-only

I know about --color-only. But it's a compromise since it only styles +/- lines, and not things like file and hunk headers. diff-so-fancy manages to retain all of those in "patch mode" (this is a fairly recent improvement):

[hahuang65]

Wanted to chime in here saying that, this seems to work for my git repositories, but for a git repository with sub-modules, git add -p is busted with the same error message.

[dandavison]

Thanks @hahuang65 that's a regression in v0.9.0. Let's discuss in https://github.com/dandavison/delta/issues/755. I'm aiming to release a fix for it shortly.

[joefiorini]

I was using an old version of delta (0.8.2) and today upgraded to 0.15.1. I've been able to use git add -p just fine until now. All of a sudden I'm getting the above error. I removed the diffFilter config from [interactive] and it started working. Could this be a regression or is this still a known issue?

[HaleTom]

@joefiorini asked if there could be a regression causing this to still be an issue.

I note this issue wasn't closed when #756 was merged in.

Was there some other cause for mismatched input that #756 didn't fix?

Or is it indeed a regression?

[dandavison]

Hi all, is anyone aware of unsolved problems here? (I'm not, but I use magit rather than git add -p).

[soispha]

Yes, I have experienced problems with this recently (my delta config is here, if that helps)

[martinetd]

I can confirm this regressed "recently", I've bisected it down to e0eef7deccc7ca1c56d7c30894985c00898ef8a3

It took me a while to understand why this only happened very rarely, but looking at the commit itself it's obvious in hindsight -- that only happens when I modified some binary files in the repo...!

With that in hand, this can be reproduced as follow:

$ mkdir /tmp/test
$ cd /tmp/test
$ git init
$ cp /bin/sh a
$ echo some text > b
$ git add a b
$ git commit -m init
$ cp /bin/cp a
$ echo other test > b
$ git add -p

I'm sure there are better ways of getting git add -p's raw output (using tee as a filter or something -- EDIT: piping from git diff would apparently have worked...), but I got it from strace and it's obvious enough from the output (it's missing the "Binary files a.. differ" line):

$ printf "\33[1mdiff --git a/a b/a\33[m\n\33[1mindex 7b65664a8972..82c76465611f 100755\33[m\nBinary files a/a and b/a differ\n\33[1mdiff --git a/b b/b\33[m\n\33[1mindex 7b57bd29ea8a..4d3b8c11a4a2 100644\33[m\n\33[1m--- a/b\33[m\n\33[1m+++ b/b\33[m\n\33[36m@@ -1 +1 @@\33[m\n\33[31m-some text\33[m\n\33[32m+\33[m\
diff --git a/a b/a3[m\n"
index 7b65664a8972..82c76465611f 100755
Binary files a/a and b/a differ
diff --git a/b b/b
index 7b57bd29ea8a..4d3b8c11a4a2 100644
--- a/b
+++ b/b
@@ -1 +1 @@
-some text
+other text
$ printf "\33[1mdiff --git a/a b/a\33[m\n\33[1mindex 7b65664a8972..82c76465611f 100755\33[m\nBinary files a/a and b/a differ\n\33[1mdiff --git a/b b/b\33[m\n\33[1mindex 7b57bd29ea8a..4d3b8c11a4a2 100644\33[m\n\33[1m--- a/b\33[m\n\33[1m+++ b/b\33[m\n\33[36m@@ -1 +1 @@\33[m\n\33[31m-some text\33[m\n\33[32m+\33[m\33[32mother text\33[m\n" | delta --color-only
diff --git a/a b/a
index 7b65664a8972..82c76465611f 100755
diff --git a/b b/b
index 7b57bd29ea8a..4d3b8c11a4a2 100644
--- a/b
+++ b/b
@@ -1 +1 @@
-some text
+other text

lunchbreak ended so I'll leave the rest to whoever cares, I've downgraded for now :)

[michaelblyons]

missing the "Binary files … differ" line

Nice find. There's also a way to dump encoded binary hunks to the diff output with --binary. Someone might want to try that out, too. (On a phone and can't try it right now.)

[dandavison]

@imbrish just in case you want to continue your great work in this area, this bug seems like it might be in your area of expertise? But no worries at all if you don't have time!

[HaleTom]

Thanks very much, @dandavison! 🥳