`lzma` import failed in Python 3.13

[kelle]

Description of the problem:

Issue is discussed here: https://github.com/pyspextool/pyspextool/issues/264

We're having a trouble with lzma import but only in Python 3.13.

File "/home/aburgasser/.local/lib/python3.13/site-packages/pooch/processors.py", line 15, in
import lzma
File "/usr/local/lib/python3.13/lzma.py", line 27, in
from _lzma import *
ModuleNotFoundError: No module named '_lzma'

System information

Python3.13 on linux, no conda environment

pooch is up to date (v2.3)

[leouieda]

@kelle thanks for reporting but this doesn't seem to come from us. We test on Python 3.13 on all platforms and things work fine. Also, the error is from an internal module of the builtin lzma Python module and not Pooch itself. I'd suggest looking into the users Python install. Seems to me like lzma is trying to import a compiled module _lzma (this is a common pattern for modules with C extensions) and it's failing because the module is not there. I don't see how this could happen on a standard Python install. So maybe this user installed Python in an unconventional way.

[dopplershift]

Just for reference, I think it's possible to compile Python without support for lzma, so that may be what's going on here. Looks like it wouldn't be too hard to make the processors populate supported compression based on what imports successfully. That said, I'm not sure how common that configuration is (or what linux distro might be doing it) or if it's worth the effort/complexity at all.

[leouieda]

Thanks for the extra input @dopplershift! @kelle if that's the case here, let me know. We can make a change here to guard those imports in try: except: to make sure things work even in these corner cases.

[kelle]

It is also not clear to me how common this situation is. I will say, it's probably faster for y'all to add the try/except than for us to track down how to get the lzma library installed. On the other hand, it's also straight forward for this user to use Python 3.12 instead of 3.13. So, whatever y'all want to do!

[harveyormston]

This has affected me as well. In my case, I'm using a custom-compiled version of Python 3.9 on Red Hat Enterprise Linux. No official package is available for that specific version of Python on that specific version of RHEL so I've had to build it myself.

My environment does not have the devel packages for lzma and bz2 installed. lzma and bz2 are both optional features when building Python. So I've compiled Python without them. I imagine my use case is not very unusual.

Since many features of pooch are quite usable without needing lzma and bz2, a fix which wrapped the imports of lzma and bz2 in a try/except would be ideal for me.

[leouieda]

👍🏾 thanks for reporting @harveyormston! It helps to know that this is something that’s affecting people.