litellm [Bug]: Fail to start LiteLLM Proxy server

What happened?

I run the command 'litellm' in command prompt and error returned. The server cannot start.

Relevant log output

Traceback (most recent call last):
  File "/home/krystal/miniconda3/envs/litellm/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 146, in <module>
    import litellm.proxy.enterprise as enterprise
ModuleNotFoundError: No module named 'litellm.proxy.enterprise'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/krystal/miniconda3/envs/litellm/lib/python3.11/site-packages/litellm/proxy/proxy_cli.py", line 238, in run_server
    from .proxy_server import (
  File "/home/krystal/miniconda3/envs/litellm/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 149, in <module>
    import enterprise  # type: ignore
    ^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'enterprise'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/krystal/miniconda3/envs/litellm/bin/litellm", line 8, in <module>
    sys.exit(run_server())
             ^^^^^^^^^^^^
  File "/home/krystal/.local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/krystal/.local/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/krystal/.local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/krystal/.local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/krystal/miniconda3/envs/litellm/lib/python3.11/site-packages/litellm/proxy/proxy_cli.py", line 250, in run_server
    from proxy_server import (
ModuleNotFoundError: No module named 'proxy_server'

Twitter / LinkedIn details

No response

Mar 15 '24 07:03 canytam-krystal

I can confirm this. Installing v1.31.8 is a workaround for now: pip install litellm[proxy]==1.31.8 Likely this bug was introduced in 1.31.9

Mar 15 '24 08:03 nherment

Thanks.

Mar 15 '24 09:03 canytam-krystal

looking into this

Mar 15 '24 14:03 ishaan-jaff

Update: I only see this on the cli but not on the docker builds

Mar 15 '24 14:03 ishaan-jaff

Able to repro and working on a fix, will patch ASAP

Mar 15 '24 14:03 ishaan-jaff

patched here: https://github.com/BerriAI/litellm/commit/fa1676b253fa8f861e57cc6b031581c270cf9497

should be fixed on 1.31.14 @nherment @canytam-krystal

Mar 15 '24 15:03 ishaan-jaff

@nherment, @canytam-krystal I'd love to hop on a call and learn how you're using litellm proxy and how we can improve for your use case - sharing my cal here for your convenience: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Mar 15 '24 15:03 ishaan-jaff

@nherment @canytam-krystal is this fixed for you ?

Mar 16 '24 04:03 ishaan-jaff

Thanks, everything is ok now.

Mar 18 '24 04:03 canytam-krystal

I can confirm this. Installing v1.31.8 is a workaround for now: pip install litellm[proxy]==1.31.8 Likely this bug was introduced in 1.31.9

Thank you! 1.31.8 works great.

Mar 21 '24 08:03 cwegener