luigi icon indicating copy to clipboard operation
luigi copied to clipboard

luigi server fails to start with python 3.12

Open EugeneYushin opened this issue 8 months ago • 3 comments

Luigi claims it supports python3.12, but fails to start central scheduler at runtime due to deprecated and removed package pkg_resources.

> mkvirtualenv --python=3.12 luigi
> python --version
Python 3.12.9
> pip install luigi
Collecting luigi
  Using cached luigi-3.6.0.tar.gz (1.2 MB)
...
> luigid
Traceback (most recent call last):
  File "/Users/eyushin/.virtualenvs/luigi/bin/luigid", line 8, in <module>
    sys.exit(luigid())
             ^^^^^^^^
  File "/Users/eyushin/.virtualenvs/luigi/lib/python3.12/site-packages/luigi/cmdline.py", line 13, in luigid
    import luigi.server
  File "/Users/eyushin/.virtualenvs/luigi/lib/python3.12/site-packages/luigi/server.py", line 47, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

EugeneYushin avatar May 05 '25 13:05 EugeneYushin

Can you try installing/upgrading these two:

pip install -U setuptools
pip install -U distribute 

aliavni avatar May 15 '25 12:05 aliavni

Sure, this is the thing with deprecated setuptools in py3.12 originally. I'd assume luigi codebase to bring all the dependencies itself when declaring it's been tested against particular py version. It might be worth refactoring usages of this deprecated API instead.

EugeneYushin avatar May 15 '25 12:05 EugeneYushin

I have a PR here. Let's see how it goes

aliavni avatar May 15 '25 13:05 aliavni