langcorn Add websocket support

To support chat-like langchain applications we need state.

Fast API sessions don't seem very straightforward, so websockets may be the way to go: https://fastapi.tiangolo.com/advanced/websockets/

See also https://github.com/hwchase17/chat-langchain for a reference on how it could be done.

Apr 25 '23 15:04 pors

How about SSE support as well?

In a LLM chat scenario, tokens only need to be streamed from server to client, not the other way around, so SSE might work better than WebSockets, especially in a serverless environment where long-running sessions are difficult to keep alive.

Apr 27 '23 06:04 andrictham

Hi all. WS support requires more invasive changes than I expected. Nevertheless, passing the memory with the request to maintain the state is possible. I made these changes available at https://github.com/msoedov/langcorn/pull/10

May 10 '23 15:05 msoedov