agents
agents copied to clipboard
livekit
[Question] Inquiry on Best Practices for Deploying Stateful Agent-Services
I would like to consult about the best practices for deploying services, particularly in scenarios involving stateful agent-services.
Currently, I have a program running, and I have added some new features. The traditional approach for stateless services would be to start a new instance of the program, redirect the traffic to the new instance, and then shut down the old one. This ensures a seamless transition for users.
However, I am working with a worker that maintains an agent, which caches conversation history and is thus a stateful service. What would be the recommended deployment practices in this case to ensure a smooth update without disrupting the user experience? Any insights or strategies would be greatly appreciated!
we have a guide on deployment here. Please have a look at let us know if you have additional questions.
specifically the upgrade path is outlined here
tl;dr is that it should be automatic, as long as you are handling terminationGracePeriod
we have a guide on deployment here. Please have a look at let us know if you have additional questions.
specifically the upgrade path is outlined here
I have read the documentation, but I need to ask a few questions due to my limited understanding. I hope you can be patient with me.
-
If I run the application similar to the one in this GitHub link, will it allow only one person to use the agent? If I want to support 20 users with agents, do I need to execute python main.py 100 times? If the line cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint)) starts a single worker and does not support auto-scaling, how can I manage this?
-
I believe there is a way for the livekit-server to identify which workers should not be assigned tasks. When a developer sends a termination signal to the agent application, the livekit-server would remove those workers from the application. Is my understanding correct? I know that auto-scaling is straightforward when using container orchestration tools like Kubernetes. However, for various reasons, I cannot use Kubernetes right now and am instead using lighter process management tools like Supervisor. I think there must be a deployment method compatible with these process management tools. Aside from sending a termination signal, how can we notify the livekit-server to remove some workers? If this is possible, I can start new applications, notify the livekit-server to remove old workers, wait for some time, and then upgrade the old applications.