kubectl-ai icon indicating copy to clipboard operation
kubectl-ai copied to clipboard

Published 20 hours ago • verified publisher icon sozercan

textgen openai compatible endpoind

Open myrulezzz opened this issue 1 year ago • 2 comments

Hi i set my export OPENAI_ENDPOINT=http://127.0.0.1:5000/v1 as i am running a model throw textgen ui but when i execute kubectl ai "create an nginx deployment with 3 replicas" I get error ⣾ Processing...Error: error, status code: 500, message: invalid character 'I' looking for beginning of value

myrulezzz avatar Dec 28 '23 07:12 myrulezzz

@myrulezzz what version of kubectl-ai are you using? v0.0.11 added support for custom endpoints

I am not sure about textgen ui, but kubectl-ai works with https://github.com/sozercan/aikit or https://github.com/mudler/LocalAI which provides a drop-in openai compatible api

sozercan avatar Dec 29 '23 19:12 sozercan

Hi i am using kubectl-ai version 0.0.11. I am trying also with huggingface text inference endpoint and it does not work. same endpoint works with k8sgpt

myrulezzz avatar Dec 29 '23 20:12 myrulezzz