llm-engine
llm-engine copied to clipboard
Add configurable queue_message_timeout_duration parameter in endpoint service configs
Pull Request Summary
What is this PR changing? Why is this change being made? Any caveats you'd like to highlight? Link any relevant documents, links, or screenshots here if applicable.
Test Plan and Usage Guide
How did you validate that your PR works correctly? How do you run or demo the code? Provide enough detail so a reviewer can reasonably reproduce the testing procedure. Paste example command line invocations if applicable.
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
| Diff | Package | Supply Chain Security |
Vulnerability | Quality | Maintenance | License |
|---|---|---|---|---|---|---|
| gunicorn@21.2.0 | ||||||
| flask@2.3.3 |