ENH/DEP: use a more robust, external implementation of the Line Integral Convolution (LIC) algorithm

[neutrinoceros]

PR Summary

I've been spending some time rewriting the Line Integral Convolution (LIC) algorithm in Rust and published it as a new, minimal, Python package: rLIC Because I wanted to see how it compared with yt's builtin Cython extension, in terms of performance, I ran a couple tests, and found that rLIC was not only much faster[^1] (more than x2):

this benchmark shows the execution time of convolving a 512 x 512 image with various kernel sizes

... it also seems to be more correct. See this very simple test case, where yt's implementation appears to be completely broken:

This is the one example from our docs, as of yt 4.4.0

and here it is with this branch

so, instead of fixing yt's implementation, I would like to propose yt adopts rLIC as a dependency.

Since yt's implementation doesn't appear to be correct, it is no surprise that rLIC isn't a drop-in replacement: new results are not identical. Bug-for-bug backward compatibility being excluded, I also made a couple adjustments, like replacing the legacy prng API that's discouraged in numpy.

Important notes:

• rLIC wheels are abi3, which means that the package is already future compatible with Python 3.14+ and will remain so until CPython breaks abi3 and creates an abi4. If this happens, I will publish new wheels. The point being that depending on rLIC is not expected to cause any disturbance when adding support for newer Python versions in yt
• I support all platforms that numpy 2.2 supports
• ~rLIC is not yet distributed through conda-forge, which I'm aware is a requirement for any dependency of another package that wants to be shipped there. I don't mind putting the package on conda-forge too, pending acceptance of this proposal~ update: nevermind I'm going for it anyway https://github.com/conda-forge/staged-recipes/pull/29385 update 2: it's up on conda-forge

PR Checklist

• [ ] New features are documented, with docstrings and narrative docs
• [ ] Adds a test for any bugs fixed. Adds tests for new features.

[^1]: if you're wondering, no, it's not just because my version is in Rust. I did spend time optimizing the implementation, in particular I avoided wasteful speculative executions which are extremely common with the naive implementation of this algorithm.

[neutrinoceros]

(I'm putting this on the 4.5.0 milestone, mostly as to mean I don't want it to be backported, but it shouldn't block any particular release)

[matthewturk]

whoa.

[neutrinoceros]

@olebole, do you have any objection to, or questions about adding this dependency to yt ? Would you expect it could be a burden to package rlic for debian ?

[olebole]

@neutrinoceros I will not be able to package it for the upcoming Debian "Trixie" release which will come summer/autumn of this year, but for later this should be possible. However, this would be my first package that uses Rust, so I am totally unfamiliar with its ecosystem, so expect some trivial questions during the packaging :grin:

[neutrinoceros]

Happy to go down this road with you, it is also my first experience mixing rust and Python, so I'm genuinely curious to find how easy or difficult the overall process turns out !

[neutrinoceros]

Let's see how it plays out over at conda-forge first https://github.com/conda-forge/staged-recipes/pull/29385

[neutrinoceros]

conda-forge is done. @matthewturk, you sounded pretty enthusiast, do you think this can go in yt 4.5.0 ?

[neutrinoceros]

power cycling this to refresh CI

[neutrinoceros]

Moving to rLIC should soon unblock the missing feature for LIC: compat with periodic datasets https://github.com/neutrinoceros/rlic/pull/196

[neutrinoceros]

@matthewturk Gentle reminder

[matthewturk]

Oh! I am sorry that I was blocking on this. I'm up for it going in, and appreciate you taking the time to implement it.