sglang icon indicating copy to clipboard operation
sglang copied to clipboard

Modify metrics service endpoint

Open lambert0312 opened this issue 2 weeks ago • 3 comments

Motivation

When using sglang to deploy LLM in the K8s environment, use the --enable-metrics parameter to open the /metrics endpoint. If the api-key is configured, it will cause problems in obtaining monitoring data.

Modifications

Similar to /health, exclude the /metrics endpoint from verifying the api-key

Checklist

  • [x] Format your code according to the Code Formatting with Pre-Commit.
  • [ ] Add unit tests as outlined in the Running Unit Tests.
  • [ ] Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
  • [ ] Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
  • [ ] For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.

lambert0312 avatar Feb 10 '25 01:02 lambert0312