sglang
sglang copied to clipboard
Modify metrics service endpoint
Motivation
When using sglang to deploy LLM in the K8s environment, use the --enable-metrics
parameter to open the /metrics
endpoint. If the api-key is configured, it will cause problems in obtaining monitoring data.
Modifications
Similar to /health
, exclude the /metrics
endpoint from verifying the api-key
Checklist
- [x] Format your code according to the Code Formatting with Pre-Commit.
- [ ] Add unit tests as outlined in the Running Unit Tests.
- [ ] Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
- [ ] Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
- [ ] For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.