aphrodite-engine icon indicating copy to clipboard operation
aphrodite-engine copied to clipboard

[Feature]: xtc sampling support for kai api

Open BlairSadewitz opened this issue 5 months ago • 3 comments

🚀 The feature, motivation and pitch

I use the koboldai webui a lot. Koboldcpp supports xtc (and dry rep pen, and maybe some other samplers) that I don't think this does. You do have xtc in the openai api, though. I'd just use the webui with the openai api endpoint, but it doesn't seem to do streaming. Am I doing something wrong?

Thanks.

Alternatives

No response

Additional context

No response

BlairSadewitz avatar Sep 25 '24 15:09 BlairSadewitz