Apoorva Kulkarni comments

Results 65 comments of


                                            Apoorva Kulkarni

[Inference]: RayLLM pattern for LLMs

Look into vLLM under the hood for autoscaling, continuous batching basically efficiently scaling LLM inference. Use https://github.com/ray-project/llmperf for benchmarking.

chore: Modified the eks version to 1.29 as defined in the issue #520

Hi @manjarisri, thanks for the PR! Have you done any tests to make sure these stacks come up without any issues?

Granting S3 access to karpenter nodes

>Thanks for the great examples! I altered the jupyterhub on eks example (for a private cluster accessed via a Tailscale VPN) and I'm now adding a ray cluster and trying...

failed calling webhook "mservice.elbv2.k8s.aws"

This is due to a mutating webhook introduced for LBC v2.5+. Per the docs... >The AWS LBC provides a mutating webhook for service resources to set the spec.loadBalancerClass field for...

feature request: elastic admin team creation without relying on system:master

I think this is a reasonable request. I will add it to our backlog.