text-generation-inference
text-generation-inference copied to clipboard
Allow specifying `adapter_id` on `chat/completions` requests
trafficstars
Feature request
It seems that if i want to load a base model with an adapter and consume it, i'll have to use the generate route only which allows specifying adapter_id
`curl 127.0.0.1:3000/generate
-X POST
-H 'Content-Type: application/json'
-d '{
"inputs": "Was "The office" the funniest tv series ever?",
"parameters": {
"max_new_tokens": 200,
"adapter_id": "tv_knowledge_id"
}
}'
but can't use v1/chat/completions
are you planing to support this?
Motivation
Many use v1/chat/completions and train lora adapters for it
Your contribution
Maybe, if you're over your capacity