Basic diff output

[nc7s]

As proposed in https://github.com/chmln/sd/pull/208#issuecomment-1763168340, produce diffs of changes, instead of printing entire files.

This is a rough WIP that copies the terminal-inline example of similar and barely works. I intend to make it produce patches that are actually applicable.

example diff (note that coloring is provided by bat)

[CosmicHorrorDev]

Thanks for the PR! I really like how the output looks. I've gone ahead and marked this as a draft while this is still a WIP, but just let me know if you need anything

As for early feedback I think my only concern is that it shouldn't replace the existing output, but should be added as another option. Maybe through a --preview-diff flag or by changing --preview to take take a value like --preview <plain|diff>?

[nc7s]

Thanks for the PR! I really like how the output looks. I've gone ahead and marked this as a draft while this is still a WIP, but just let me know if you need anything

Didn't know you can do that ;)

As for early feedback I think my only concern is that it shouldn't replace the existing output, but should be added as another option. Maybe through a --preview-diff flag or by changing --preview to take take a value like --preview <plain|diff>?

Indeed, the word preview has a meaning. Maybe just --diff? Since with such a flag it seems semantically incorrect to overwrite the file anyway.

[CosmicHorrorDev]

Sure --diff seems good!

[nc7s]

Early warning: I'm tempted to (and in fact in the process of) refactoring the code with modular input/output/replace. @CosmicHorrorDev, please stop me early if you hesitate to see that.

[CosmicHorrorDev]

Hmmm, I think I would be all right with that. Just push changes and ping me whenever you want feedback

I am planning on pushing for a release over the next few days

[nc7s]

It works ™. Although, it lacks a context radius argument. I tend to not use a generic name like --context for a diff. But --diff-context is too long. I also don't know how to name the "file" when input is through stdin, so it's currently empty; maybe the user knows better, so we also provide an argument like --diff-filename?

[alexilyaev]

Can you share an example of how it would look like?

Got here after searching how to make --preview only show the matched strings or lines that would change and not print the whole file.

Example

This will print every single line of the file and then only at the end prints the change in color. In large files, this makes --preview unusable.

echo "111\n222\n333\n444\n555\n666\n777\n888\n999" | sd -p '(999)' '000'

So, I'm wondering if the diff mode will help with that scenario?

Similar open issues

I suppose one of these would cover the large file use case as well

• https://github.com/chmln/sd/issues/150
• https://github.com/chmln/sd/issues/164

[nc7s]

Can you share an example of how it would look like?

You can click the triangle on the left of "example" in the topic. Later updates differ a little in detail, but largely it's the same. The Wikipedia entry on diff is probably also worth a read.

Got here after searching how to make --preview only show the matched strings or lines that would change and not print the whole file.

So, I'm wondering if the diff mode will help with that scenario?

Probably. I never thought of diffs in this perspective, but it does work in this way (showing only changes, and limited surrounding context).

[alexilyaev]

@nc7s Oh, didn't notice the ▶ next to the example diff ☺️👍🏼 How did you get it to show colors? (I'm on macOS, using Zsh in iTerm)

Ok, so I'm testing -d with a change on consecutive lines and it does show me only the changed lines. But, it doesn't show me the change for each line, rather a big chunk of "before" lines and then a big chunk of "after" lines:

echo "123\n123\n123\n123\n123\n123\n123\n123\n123\n123\n" | cargo run -- -d '2' '9'

If the changes are not in consecutive line, then it works as expected:

echo "123\n000\n123\n000\n123\n000\n123\n000\n123\n000\n" | cargo run -- -d '2' '9'

I still think that -p should have a flag to show only the affected lines and not print the whole file.

[nc7s]

@alexilyaev:

How did you get it to show colors? (I'm on macOS, using Zsh in iTerm)

bat. Now spelled in the <summary>.

Ok, so I'm testing -d with a change on consecutive lines and it does show me only the changed lines. But, it doesn't show me the change for each line, rather a big chunk of "before" lines and then a big chunk of "after" lines:

I still think that -p should have a flag to show only the affected lines and not print the whole file.

This is expected behavior of diffs. Recommended read: https://en.wikipedia.org/wiki/Diff

Sometimes you won't know where and what the changed lines are if no surrounding context is provided. That's why diffs have contexts - to match surrounding lines. They may otherwise update an unrelated but identical line, which is totally possible for short lines of common words.

If you are confident, context radius could be reduced, down to zero. I'm considering an option to set that, but haven't decided on its naming yet.

@CosmicHorrorDev: copied over from earlier, maybe covered in noise:

Although, it lacks a context radius argument. I tend to not use a generic name like --context for a diff. But --diff-context is too long. I also don't know how to name the "file" when input is through stdin, so it's currently empty; maybe the user knows better, so we also provide an argument like --diff-filename?

[chmln]

This is great as a prototype, excellent work. I have long wanted to implement something like this.

Not sure if this is actually possible but it would be great if we could integrate one of the rust pagers as a library so that users don't necessarily need to have e.g. bat installed and piped into, to get a nice looking diff output

[nc7s]

@chmln:

Not sure if this is actually possible but it would be great if we could integrate one of the rust pagers as a library so that users don't necessarily need to have e.g. bat installed and piped into, to get a nice looking diff output

My thought then was that it feels unnecessary for every CLI program to implement its own colorization, esp. when the output is in a pretty common format. Feels like bloat.

[CosmicHorrorDev]

For me the big thing is that an integrated pager can be more aware of its contents. An external pager is usually very naïve when it comes to searching or even determining terminal width

[chmln]

My thought then was that it feels unnecessary for every CLI program to implement its own colorization, esp. when the output is in a pretty common format.

That's a valid point, although somewhat more of a philosphical one. Ultimately, the users want to see a pretty diff. If we can make that happen without having them install bat and configure a good looking theme..I think that's a clear win.

And if someone really wants to use their own diff program, we can detect that our diff output is being piped and just produce the raw version as you suggest.

Addendum: nevertheless, shipping this 'raw' diff mode is still a big win for all the power users. Just wanted to provide some ideas and food for thought.

[CosmicHorrorDev]

Also sorry for taking so long to give feedback. I've been stretched pretty thin lately. I likely won't have time to take a look till the new year

[nc7s]

@CosmicHorrorDev:

For me the big thing is that an integrated pager can be more aware of its contents. An external pager is usually very naïve when it comes to searching or even determining terminal width

I'm no expert in this field, but fear that properly setting up an integrated one requires serious work (you gotta know the ins and outs). Plus, we've already seen GIT_PAGER and QUILT_PAGER, to name a few.

Also sorry for taking so long to give feedback. I've been stretched pretty thin lately. I likely won't have time to take a look till the new year

No worries ;) we all have a bunch of work at year end.

@chmln:

My thought then was that it feels unnecessary for every CLI program to implement its own colorization, esp. when the output is in a pretty common format.

That's a valid point, although somewhat more of a philosphical one. Ultimately, the users want to see a pretty diff. If we can make that happen without having them install bat and configure a good looking theme..I think that's a clear win.

Diff coloring is almost too easy to not implement, certainly can be made to work. The "do one thing" principle sometimes has a vague boundary…

And if someone really wants to use their own diff program, we can detect that our diff output is being piped and just produce the raw version as you suggest.

Yeah, configurable colorization was talked about in https://github.com/chmln/sd/issues/268.

So, combined, maybe we can have colorless diffs first, then implement proper colorization later in one go.

[oriongonza]

Hello @chmln, @nc7s

With @CosmicHorrorDev stepping down what's the status on this? I think it's possitive if either noctis or me would have merge permissions. As I said before don't want to wholely maintain but I wouldn't mind reviewing PRs and doing basic fixes at least until another maintainer shows up.