copilot icon indicating copy to clipboard operation
copilot copied to clipboard
verified publisher icon openchatai

Make socket io sessions scale

Open gharbat opened this issue 1 year ago • 1 comments

Currently, the socket connections work fine when there is a single LLM server instance. Once we have multiple instances, the socket connections get lost because they are not sticky, and the load balancer routes requests to a server where the session ID is not defined.

gharbat avatar Feb 14 '24 00:02 gharbat

Hey, I want to work on this. Can you assign this to me?

itssubhodiproy avatar Feb 14 '24 05:02 itssubhodiproy