error when running w/ python 3.10: old numpy is used

[phiweger]

Trying to run

lmql serve-model gpt2-medium --cuda

on python 3.10, but this fails bc/

# ImportError: cannot import name 'Sequence' from 'collections'

This is because since python 3.10 you need to import Sequence from collections.abc not collections

python3.10 -c "from collections.abc import Sequence"
# works
python3.8 -c "from collections import Sequence"
# works, but:
# DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working

[lbeurerkellner]

Can you share more about your environment, e.g. can you provide the version of transformers in use. LMQL does not import 'Sequence' from 'collections' anywhere in its code base, so I suspect there may be an issue in one of the libraries.

[phiweger]

sure:

the env is a vanilla colab instance w/ python 3.10, I run nothing but:

!pip install lmql
!lmql serve-model gpt2-medium --cuda

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/numpy/core/__init__.py", line 23, in <module>
    from . import multiarray
  File "/usr/local/lib/python3.10/dist-packages/numpy/core/multiarray.py", line 10, in <module>
    from . import overrides
  File "/usr/local/lib/python3.10/dist-packages/numpy/core/overrides.py", line 8, in <module>
    from numpy.compat._inspect import getargspec
  File "/usr/local/lib/python3.10/dist-packages/numpy/compat/__init__.py", line 12, in <module>
    from . import py3k
  File "/usr/local/lib/python3.10/dist-packages/numpy/compat/py3k.py", line 21, in <module>
    from pathlib import Path
  File "/usr/local/lib/python3.10/dist-packages/pathlib.py", line 10, in <module>
    from collections import Sequence
ImportError: cannot import name 'Sequence' from 'collections' (/usr/lib/python3.10/collections/__init__.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 187, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/lib/python3.10/runpy.py", line 110, in _get_module_details
    __import__(pkg_name)
  File "/usr/local/lib/python3.10/dist-packages/lmql/__init__.py", line 17, in <module>
    import lmql.runtime.lmql_runtime as lmql_runtime
  File "/usr/local/lib/python3.10/dist-packages/lmql/runtime/lmql_runtime.py", line 9, in <module>
    from lmql.ops.ops import *
  File "/usr/local/lib/python3.10/dist-packages/lmql/ops/__init__.py", line 1, in <module>
    from lmql.ops.follow_map import *
  File "/usr/local/lib/python3.10/dist-packages/lmql/ops/follow_map.py", line 3, in <module>
    from lmql.ops.token_set import *
  File "/usr/local/lib/python3.10/dist-packages/lmql/ops/token_set.py", line 8, in <module>
    from lmql.utils import nputil
  File "/usr/local/lib/python3.10/dist-packages/lmql/utils/nputil.py", line 1, in <module>
    import numpy as np
  File "/usr/local/lib/python3.10/dist-packages/numpy/__init__.py", line 144, in <module>
    from . import core
  File "/usr/local/lib/python3.10/dist-packages/numpy/core/__init__.py", line 49, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.10 from "/usr/bin/python3"
  * The NumPy version is: "1.22.4"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: cannot import name 'Sequence' from 'collections' (/usr/lib/python3.10/collections/__init__.py)

import numpy; numpy.__version__
# 1.22.4 < seems right
!python --version
# Python 3.10.12 < also as expected

@lbeurerkellner thank you for looking into this

[lbeurerkellner]

Looks like we are a bit too loose with our dependency requirement on bumpy, and an old version is incompatible.

Could you try installing lmql with:

pip install lmql[hf] numpy==1.25.1

This fixes the issue on my end when running with Colab. If you can confirm this working on your end, I will adjust LMQL's project dependency file accordingly.

For what it is worth, here is my playground in Colab: https://colab.research.google.com/drive/154xVFkGL54sn7gMEO57G-eOQkbkHWiCZ?usp=sharing

[lbeurerkellner]

Experimenting more with lmql serve-model I found that another issue may be causing problems here. I can advise to experiment with the solution shown in the colab above, and I will further investigate how we can get lmql serve-model running in a colab environment. It is a bit unusual, since Colab Notebooks typically are not made for multi-process workflows.