aphrodite-engine
aphrodite-engine copied to clipboard
[Feature]: xtc sampling support for kai api
🚀 The feature, motivation and pitch
I use the koboldai webui a lot. Koboldcpp supports xtc (and dry rep pen, and maybe some other samplers) that I don't think this does. You do have xtc in the openai api, though. I'd just use the webui with the openai api endpoint, but it doesn't seem to do streaming. Am I doing something wrong?
Thanks.
Alternatives
No response
Additional context
No response