Tyler Yang issues

Repositories
Issues
Comments

Results 3 issues of


                                            Tyler Yang

fake client returns no kind "ClusterServingRuntimeList" is registered for version "serving/v1alpha1"

/kind bug What steps did you take and what happened: [A clear and concise description of what the bug is.] I'm using fake client using NewSimpleClientset and testing List functionality...

kind/bug

Prometheus metric support for OpenAI endpoints

/kind feature **Describe the solution you'd like** Expose metrics for the OpenAI endpoints for models using the huggingfaceserver runtime. The following metrics to be exposed should include: TTFT (Time to...

Huggingface runtime (hf backend) fails to return for all following calls if wrong endpoint is hit

/kind bug **What steps did you take and what happened:** I am following through the steps of deploying the google-t5/t5-small in an AWS cluster with KServe v0.13.1. The model deploys...