text-generation-webui icon indicating copy to clipboard operation
text-generation-webui copied to clipboard

Published 20 hours ago • verified publisher icon oobabooga

Add stop tokens to API

Open moorehousew opened this issue 1 year ago • 3 comments

Pretty much what it says on the tin- it'd be nice to be able to pass an array of strings to the API which will terminate a completion early if they're detected in the output.

moorehousew avatar Mar 10 '23 19:03 moorehousew

Yes +1 on this for sure.

DavesDevFails avatar Mar 10 '23 21:03 DavesDevFails

I think that this can be done through an extension that creates an additional API endpoint. I will try to do it later.

oobabooga avatar Mar 10 '23 21:03 oobabooga

Thanks... is there any way to set a stop token / sequence at the moment in the webui? I can't seem to see anything.

DavesDevFails avatar Mar 10 '23 21:03 DavesDevFails

This is now implemented in the API extension https://github.com/oobabooga/text-generation-webui/pull/744

oobabooga avatar Apr 03 '23 16:04 oobabooga