🚀 Using delta as library

[extrawurst]

This library looks great! Is it possible to use it as a crate in another rust project? gitui could really benefit from such state of the art diff visualisation

[Kr1ss-XD]

Nice one ! That would be great.

[dandavison]

Hi @extrawurst, thanks! Delta isn't currently structured as a library for Cargo, but shall we sketch out what the minimal version of this could look like? The delta test suite already contains a function that takes an input diff as a string, and a delta config struct, and returns delta output as a string. So, as a quick first thought, If we exposed functions like these, is there any chance that that could suffice as a minimal first version of the library? https://github.com/dandavison/delta/blob/6912b4df9507ca43fdcacbaf4b9e07fdb6a26646/src/tests/integration_test_utils.rs#L74-L84

https://github.com/dandavison/delta/blob/6912b4df9507ca43fdcacbaf4b9e07fdb6a26646/src/tests/integration_test_utils.rs#L51-L53

What sort of API would you like to see there?

[extrawurst]

@dandavison ok I might need more context here. what kind of input would that require? and what output does it generate?

[dandavison]

input would be a string containing the stdout of a git diff command (git diff, git show, git log -p, git stash show -p etc) or diff -u. The return value would be a new string, representing the same diff as the input string, but with ANSI color escape sequences inserted to effect delta's coloring, and possibly with additional decoration elements (boxes, lines, reconfigured hunk header, side-by-side layout, line numbers). The details of exactly which colors and decoration elements would be applied to the input would be determined by the config struct. Basically every option mentioned in the delta README would be configurable by setting a value on the config struct.

[dandavison]

Incidentally, lazygit already uses delta, but as I understand it lazygit calls git as an external process whereas you are using libgit2. Is that right? So lazygit didn't need delta as a library; a delta process is just inserted in between git's output and lazygit's input.

[dandavison]

More precisely, I think the input might be three things:

1. A string containing git's diff output
2. A git2::Config struct representing the user's git config (which might have relevant delta config).
3. Some additional delta command line options (optional)

So my question at this stage is: would an API something like this give you what you need?

/// Run delta on `git_output` (a string containing a diff in git/unified diff format), respecting
/// relevant configuration in `git_config` and `command_line_options`. Return a string containing
/// the diff as represented by delta: i.e. with colors and optional decorations, optionally in
/// side-by-side layout, with line-numbers, etc, as specified by `git_config` and
/// `command_line_options`.
pub fn delta(git_output: &str, git_config: &git2::Config, command_line_options: &[&str]) -> String {
}

[dandavison]

@extrawurst What would a minimal API look like that would be useful for your purposes?

[jeroen]

I would be very interested in this if you can export headers that we can call from C. A good example is gifski: https://github.com/ImageOptim/gifski/blob/main/gifski.h

[extrawurst]

@dandavison i guess I would need to be able to pass a buffer a and buffer b (I would get them straight out of git2-rs and then return me some kind of datastructure that allows me to render the lines correctly - I guess I am flexible on this end

[Cldfire]

Hi! I just came by the repo looking for exactly this--I'd love a library version that could be called from Rust code 😄

My use case would just be to take two strings of source code and diff them. If a helper could be provided to handle creating the git diff so I didn't have to, that would be even more convenient, but either way works.

[jonashaag]

I'm also interested in this. I maintain a Git web viewer written in Python, and I'd love to level up the diff rendering game by using delta, which has one of the best diff engines I've come across.

The interface of what I'm currently using is you pass old and new text: https://github.com/jonashaag/klaus/blob/3d090ec0a7ba62982b467e0f17813ba3d6a04c68/klaus/repo.py#L299-L301

And you get back a list of line changes + metadata (old line number, new line number, type of modification): https://github.com/jonashaag/klaus/blob/3d090ec0a7ba62982b467e0f17813ba3d6a04c68/klaus/diff.py#L55-L63

The format of the line changes is described here: https://docs.python.org/3/library/difflib.html#difflib.SequenceMatcher.get_opcodes

[dandavison]

Hi all, thanks for the interest here. It should definitely be possible to refactor delta as a Rust crate exposing a function that can be passed a diff and will return a string containing the same diff altered by delta and with ANSI escape sequences representing colors, and also exposing the ability to create the necessary delta config object to control that process. I'd like to do this, but not sure yet when I'll find time.

@jeroen exporting headers that can be called from C sounds interesting and I'm sure I'd learn a lot doing that.

@jonashaag your application sounds like it would involve exposing a substantially lower-level interface in delta: i.e. instead of returning a string adorned with ANSI escape sequences, you want a delta library function that returns a data structure describing the results of its within-line edit inference process. How would you interface with such a Rust function from Python? Or would you write a small Rust executable that uses delta as a library, call that as a separate process from your python code, and parse its output into the necessary python data structures?

[jonashaag]

Yes, either of the interfacing is fine as long as I don’t have to parse ANSI sequences 😀. If you export a C interface that’s even easier to interface from Python (or at least it doesn’t require any additional dependencies). Even a text based interface that’s not based on ANSI sequences would be fine.

[raphCode]

There is still interest from other projects in a library feature, e.g. https://github.com/extrawurst/gitui/issues/358#issuecomment-1029273184.

Is there anything we can do to help the process? If I read correctly we need to agree on an useful API first?

My take on this would probably be datastructures wrapping each line/changed word to indicate if it was added/removed. This seems to correspond to the implementation of https://github.com/mitsuhiko/similar, which is also a rust diff library.

An alternative "out-of-band" signalling datastructure could be also interesting, since it plays nice with separate syntax highlighting: You have a buffer of text, and some structs to annotate a certain range of bytes/chars. These annotations could be diff-related or colors for syntax highlighting. Annotation data for different purposes can be easily merged when printing to the screen, by iterating the annotations and text simultaneously.

[th1000s]

For benchmarking I have done some experimental librarification in #902 and one way forward would be to leave the binary in this git-delta crate and have its version number refer to the API stability of the command line and configs.

Then most code is moved into lib-git-delta-experimental, where the "real" delta binary is present in the [[bin]] section. This crate would primarily serve git-delta for now and its version would remain 0.0.xyz while experimenting with various APIs.

For now grabbing an entire diff via delta(lines_in, &mut writer, &config) could be used by gitui. A streaming version would have to be created for larger input.

[dandavison]

Thanks @th1000s -- I'd be very happy to keep up movement on here and get the librarification refactor in #902 in. Currently the use of /// docs is very patchy and inconsistent so we might want to disable that.

[dandavison]

Please see https://github.com/dandavison/delta/pull/1013 and https://github.com/dandavison/delta/pull/1017 for the current state of librarification work. Please shout out if you're interested in working on this (AFAIK no-one currently is).

[extrawurst]

A streaming version would have to be created for larger input.

at this point streaming way to do it is not even the scope (at least for gitui, we at this point libgit2 for diffing and it also does not support a streaming version)