llm-engine icon indicating copy to clipboard operation
llm-engine copied to clipboard
verified publisher icon scaleapi

self host on runnpod

Open Stealthwriter opened this issue 2 years ago • 1 comments

Hi, can you please add documentation to self-host on runpod? it offers GPU prices cheaper than aws.

Stealthwriter avatar Sep 21 '23 10:09 Stealthwriter

Hi @Stealthwriter , we currently don't have any plans on supporting runpod. LLM Engine works on k8s, so if runpod supports k8s, then it can almost work - we'd need to find substitutes for S3.

yixu34 avatar Sep 29 '23 23:09 yixu34