Failure to import langserve because fastapi is missing

[cbornet]

trafficstars

LangServe declares fastapi as optional dependency but fastapi seems required. In a new virtual env:

pip install langserve

>>> import langserve
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/private/tmp/pythonProject/lib/python3.11/site-packages/langserve/__init__.py", line 9, in <module>
    from langserve.server import add_routes
  File "/private/tmp/pythonProject/lib/python3.11/site-packages/langserve/server.py", line 28, in <module>
    from fastapi import HTTPException, Request, Response
ModuleNotFoundError: No module named 'fastapi'

I can fix with a PR if needed.

[eyurtsev]

Hi @cbornet thanks for reporting! This behavior is intentional -- Langserve contains both server and client code, and the client code should be installable without fastapi as a requirement.

Use the 'all' extra to install both client and server dependencies.

[cbornet]

Currently fastapi is a dependency of both the server and the client:

 python -m venv /tmp/venv 
source /tmp/venv/bin/activate
pip install "langserve[client]"
python

>>> from langserve import RemoteRunnable
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/private/tmp/venv/lib/python3.11/site-packages/langserve/__init__.py", line 9, in <module>
    from langserve.server import add_routes
  File "/private/tmp/venv/lib/python3.11/site-packages/langserve/server.py", line 28, in <module>
    from fastapi import HTTPException, Request, Response
ModuleNotFoundError: No module named 'fastapi'

It is because both the server and client code are imported in the root __init__.py : https://github.com/langchain-ai/langserve/blob/main/langserve/init.py#L7-L9

[eyurtsev]

Yeah :(

It's a bug -- tests aren't set up to catch that yet, once the tests are set up should be much easier hopefully

[Vinno97]

Is there any work on this? This issue currently has a wontfix label, but the current code makes the optional client and server dependency groups required anyways.

Like @cbornet mentioned, both the client and the server are imported in the root __init__.py.

I believe this leaves three solutions:

• The root __init__.py needs to not import the submodules (luckily LangServe's API version is not yet in stable 1.x.x)
• The imports are wrapped in try-catch statements (would make server not importable when the server dependencies are not installed, but I guess that could be intended behaviour)
• The imports in the client and server code should be wrapped in if-statements/try-catch statements, with added exceptions when their logic is called. (would not change the API, would be more trouble than it's worth in my opinion)

[thaysom22]

I came across this because I want to use RemoteRunnable to invoke a runnable exposed with langserve in another service.

When I try to import it, I get this error. I'd prefer not to install fastapi as a dependency in my calling service, just to essentially make a http request, but I like the wrapper. I'm leaning towards just using requests instead of RemoteRunnable. Any suggestions?

[Vinno97]

Honestely? Just patch it yourself, it doesn't seem like they're ever going to patch this dumb design flaw. If you have control over the installations, any of the three options could work. Otherwise, you could do some very dirty import patching like this:

import sys

class fakemodule(object):
    @classmethod
    def __getattribute__(cls, name: str):
        return cls

sys.modules["fastapi"] = fakemodule()
sys.modules["fastapi.encoders"] = fakemodule()
sys.modules["fastapi.exceptions"] = fakemodule()
sys.modules["starlette"] = fakemodule()
sys.modules["starlette.requests"] = fakemodule()
sys.modules["starlette.responses"] = fakemodule()
sys.modules["fastapi.responses"] = fakemodule()

from langserve import RemoteRunnable

[Vinno97]

@eyurtsev any update on this?

[eyurtsev]

Hi @Vinno97, I won't get to this soon, but I'm happy to review any PRs that fix this issue and happy to provide guidance on how to implement it.

[Vinno97]

Sure, I wouldn't mind taking this up. However, my prefered solution (the first option from my previous message), would require an API-breaking change. Its effect would be fairly minimal as people would just need to change out their imports for more specific ones, though it's still breaking:

from langserve import RemoteRunnable -> from langserve.client import RemoteRunnable
from langserve import add_routes, APIHandler -> from langserve.server import add_routes, APIHandler

For reference, my previous suggestions:

I believe this leaves three solutions:

• The root init.py needs to not import the submodules (luckily LangServe's API version is not yet in stable 1.x.x)
• The imports are wrapped in try-catch statements (would make server not importable when the server dependencies are not installed, but I guess that could be intended behaviour)
• The imports in the client and server code should be wrapped in if-statements/try-catch statements, with added exceptions when their logic is called. (would not change the API, would be more trouble than it's worth in my opinion)

I am not aware of any method to detect the importing of a specific method. If that would be possible, we could print a deprecation notice for a version or two when people still use the from langserve import ... path, possibly making a hybrid out of option 1 and 2.