[Bug]: No module named 'proxy_server'

[mictadlo]

What happened?

conda create -n ag python=3.11
conda activate ag

pip install autogenstudio
pip install litellm --upgrade
litellm --model ollama/mistral 

Relevant log output

Traceback (most recent call last):
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/litellm/proxy/proxy_cli.py", line 238, in run_server
    from .proxy_server import (
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 11, in <module>
    import backoff
ModuleNotFoundError: No module named 'backoff'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/michal/miniconda3/envs/ag/bin/litellm", line 8, in <module>
    sys.exit(run_server())
             ^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/litellm/proxy/proxy_cli.py", line 250, in run_server
    from proxy_server import (
ModuleNotFoundError: No module named 'proxy_server'

### Twitter / LinkedIn details

_No response_

[ishaan-jaff]

Fixed here: https://github.com/BerriAI/litellm/commit/177571bea3e9990045c55251bbba1aa7c1e86d60

@mictadlo

[mictadlo]

% pip install litellm --upgrade
Requirement already satisfied: litellm in /Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages (1.31.16)

 % litellm --model ollama/mistral 
Traceback (most recent call last):
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 32, in <module>
    import backoff
ModuleNotFoundError: No module named 'backoff'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/michal/miniconda3/envs/ag/bin/litellm", line 8, in <module>
    sys.exit(run_server())
             ^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/litellm/proxy/proxy_cli.py", line 247, in run_server
    raise e
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/litellm/proxy/proxy_cli.py", line 238, in run_server
    from .proxy_server import (
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 38, in <module>
    raise ImportError(f"Missing dependency {e}. Run `pip install 'litellm[proxy]'`")
ImportError: Missing dependency No module named 'backoff'. Run `pip install 'litellm[proxy]'`

[krrishdholakia]

@mictadlo which version of litellm are you on now? I can confirm i don't see this on latest - v1.32.1

[mictadlo]

No sure,

 % litellm -v    
Traceback (most recent call last):
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 32, in <module>
    import backoff
ModuleNotFoundError: No module named 'backoff'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/michal/miniconda3/envs/ag/bin/litellm", line 8, in <module>
    sys.exit(run_server())
             ^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/litellm/proxy/proxy_cli.py", line 247, in run_server
    raise e
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/litellm/proxy/proxy_cli.py", line 238, in run_server
    from .proxy_server import (
  File "/Users/michal/miniconda3/envs/ag/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 38, in <module>
    raise ImportError(f"Missing dependency {e}. Run `pip install 'litellm[proxy]'`")
ImportError: Missing dependency No module named 'backoff'. Run `pip install 'litellm[proxy]'`

[cwegener]

Still happening on 1.32.9

[fabioxgn]

Happening in 1.33.1, I changed it to 1.32.1 and that one works.

[ZheRui-Yang]

still happening in 1.33.2

[krrishdholakia]

@mictadlo the error you're seeing says -

pip install 'litellm[proxy]'

can you run that and let me know if that solves it? The proxy has additional dependencies

[krrishdholakia]

@fabioxgn @cwegener @ZheRui-Yang which error did you see?

@mictadlo's error seems related to not running pip install 'litellm[proxy]'

[cwegener]

The error with 1.32.9 was:

ModuleNotFoundError: No module named 'proxy_server'

Why are there so many versions???

[ZheRui-Yang]

Edit:

I installed with $ pip install 'litellm[proxy]' just a moment before.

---

version 1.33.2:

ModuleNotFoundError: No module named 'proxy_server'

Full traceback:

$ litellm --test
Traceback (most recent call last):
  File "/home/zryang/src/gpt-pilot/pilot/foo-env/lib/python3.11/site-packages/litellm/proxy/proxy_cli.py", line 231, in run_server
    from .proxy_server import (
  File "/home/zryang/src/gpt-pilot/pilot/foo-env/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 109, in <module>
    from litellm.proxy.auth.handle_jwt import JWTHandler
  File "/home/zryang/src/gpt-pilot/pilot/foo-env/lib/python3.11/site-packages/litellm/proxy/auth/handle_jwt.py", line 11, in <module>
    from jwt.algorithms import RSAAlgorithm
ImportError: cannot import name 'RSAAlgorithm' from 'jwt.algorithms' (/home/zryang/src/gpt-pilot/pilot/foo-env/lib/python3.11/site-packages/jwt/algorithms.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/zryang/src/gpt-pilot/pilot/foo-env/bin/litellm", line 8, in <module>
    sys.exit(run_server())
             ^^^^^^^^^^^^
  File "/home/zryang/src/gpt-pilot/pilot/foo-env/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zryang/src/gpt-pilot/pilot/foo-env/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/zryang/src/gpt-pilot/pilot/foo-env/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zryang/src/gpt-pilot/pilot/foo-env/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zryang/src/gpt-pilot/pilot/foo-env/lib/python3.11/site-packages/litellm/proxy/proxy_cli.py", line 243, in run_server
    from proxy_server import (
ModuleNotFoundError: No module named 'proxy_server'

[wey-gu]

pi

+1, same here.

[webkonstantin]

pip install cryptography should fix it

[rick-github]

Confirm that pip install cryptography fixes it for litellm-1.33.4.

[krrishdholakia]

@webkonstantin @rick-github thanks for the find. I'll add it on our end too.

Will also add a cli test to our ci/cd to catch future regressions.

Thanks everyone!

[krrishdholakia]

Dependency fixed - https://github.com/BerriAI/litellm/commit/24204cec6b716ceacc980f874f96997aa7d7ceea

Testing added - https://github.com/BerriAI/litellm/commit/f0bee037adf595b189c0282ced91047699494bd1

If you have suggestions on how we can improve our reliability, please let me know!

[mufenzhimi]

have you solve the problem? cause I still face this problem at litellm-1.33.4

[krrishdholakia]

@mictadlo could you upgrade to v1.34.39 and check if it persists. if so, please post a stacktrace of it?

Closing issue as we now have testing for this as well