probability icon indicating copy to clipboard operation
probability copied to clipboard

Can you break your dependency on dm-tree?

Open NeilGirdhar opened this issue 1 year ago • 2 comments

DM-tree doesn't support Python 3.13, which prevents tensorflow-probability from installing on Python 3.13 and therefore all projects that depend on it.

Moreover, DM-tree appears to be unmaintained since they don't even officially support Python 3.12 (based on their pyproject trove classifiers).

Based on the commit log, it appears to be more-or-less unmaintained.

Is there any way to remove this package as a dependency? It appears you only use it one place and a lot of the functionality is already provided by jax.tree.map/flatten, etc. You may also want to look at Equinox, which is actively maintained. Or just implement the functionality in Python since speed is probably not that important, and not worth the maintenance hassle.

NeilGirdhar avatar Oct 05 '24 06:10 NeilGirdhar

Potentially we could only require dm-tree for non-JAX variants (TF and NumPy). Not sure if that solves your concern, though.

On Sat, Oct 5, 2024 at 8:55 AM Neil Girdhar @.***> wrote:

DM-tree doesn't support Python 3.13 https://github.com/google-deepmind/tree/issues/119, which prevents tensorflow-probability from installing on Python 3.13 and therefore all projects that depend on it.

Moreover, DM-tree appears to be unmaintained since they don't even officially support Python 3.12 (based on their pyproject trove classifiers).

Based on the commit log https://github.com/google-deepmind/tree/commits/master/, it appears to be more-or-less unmaintained.

Is there any way to remove this package as a dependency?

— Reply to this email directly, view it on GitHub https://github.com/tensorflow/probability/issues/1845, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFJFSI3TIB5H4UAZLQFWKKTZZ6EMNAVCNFSM6AAAAABPNCSUZCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGU3DONZQGEZTCMA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

brianwa84 avatar Oct 07 '24 13:10 brianwa84

That would work if you plan on putting dm-tree into an extra so that non-Jax variants have to install tensorflow-probability as tensorflow-probability[nonjax] or something like that. Is that what you're planning?

Why not just re-implement dm-tree in Python (which shouldn't take too long)? Do you really need this in C++? Either way, the result is Jitted.

Just guessing based on the longstanding issues in dm-tree showing the build failures, the situation with dm-tree is probably only going to get worse and I guess Tensorflow-probability will be forced to do something. What do you think?

NeilGirdhar avatar Oct 07 '24 17:10 NeilGirdhar