import mii not working

[pradeepdev-1995]

the import mii code throws the given error

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/pip_packages/mii/__init__.py", line 5, in <module>
    import grpc
  File "/tmp/pip_packages/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/tmp/pip_packages/grpc/_compression.py", line 20, in <module>
    from grpc._cython import cygrpc
  File "src/python/grpcio/grpc/_cython/cygrpc.pyx", line 27, in init grpc._cython.cygrpc
  File "/tmp/pip_packages/asyncio/__init__.py", line 21, in <module>
    from .base_events import *
  File "/tmp/pip_packages/asyncio/base_events.py", line 296
    future = tasks.async(future, loop=self)
                   ^
SyntaxError: invalid syntax

in Python 3.8.16 and CUDA Version: 12.0 and

deepspeed==0.13.1
deepspeed-kernels==0.0.1.dev1698255861
deepspeed-mii==0.2.0

[mrwyattii]

This import error is coming from grpc/asyncio. Can you please share what version you have installed for each of those? pip list | grep "asyncio\|grpc"

[pradeepdev-1995]

@mrwyattii here is the list

asyncio                  3.4.3
grpcio                   1.60.0
grpcio-tools             1.60.0
nest-asyncio             1.5.6

[mrwyattii]

@pradeepdev-1995 what version of Python are you using? I think asyncio is now part of the standard library for Python. Perhaps this older version is being used instead and causing problems? Can you try uninstalling both asyncio packages and see if that fixes the import error? pip uninstall -y asyncio nest-asyncio

[pradeepdev-1995]

Hi the pip uninstall -y asyncio nest-asyncio solved the import mii issue. But after mii importing, when I am loading my mistral finetuned model by

import mii
finetuned_model_path = "path"
pipe = mii.pipeline(finetuned_model_path)

it shows another error

RuntimeError: Ninja is required to load C++ extensions

I solved it by adding

sudo yum install ninja-build

But after ninga-build installation ,it again shows the given error in mii.pipeline(finetuned_model_path) command

subprocess.CalledProcessError: Command '['which', 'c++']' returned non-zero exit status 1.

Solved the above error by

sudo yum install gcc-c++

But now again the mii.pipeline(finetuned_model_path) command shows given error

subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
RuntimeError: Error building extension 'inference_core_ops'

[mrwyattii]

@pradeepdev-1995 can you please share the whole error output text? Can you also share your python environment (output from pip list)? Thanks!