Add `stop_regex` parameter to `/generate`

[dhbrojas]

Feature request

Add an optional stop_regex parameter to GenerateParameters. Stop generating tokens if stop_regex matches the generated text.

Motivation

Many applications such as inline code completion require fine-grained control over generated text. For example, you may want to generate a full "block" of code in which case you may desire to stop generation once a "dedent" is matched. Another example is generating till a matching a closing brace, quote or parenthesis is generated by the model. This is sometimes hard to express using TGI's "stop sequences" feature.

Your contribution

I have a working small demo, I'm open to making a PR if needed.

A few notes:

• Security and performance must be considered. Regexes from Python's re package can sometimes be very slow, especially in the presence of adversarial patterns. I think the best approach to this is to disable the feature by default (similar to how grammars work) and/or use a more restrictive Regex engine.

[dhbrojas]

Apologies, I didn't realise this was a duplicate of #1130. I see the suggested approach is to use streaming and drop the connection. Nonetheless, is it something that you would still consider?

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.