Make reverse prompt option act as a stop token in non-interactive sce…

[data-angel]

…narios. This use case is important for exposing the command line main example in non-interactive mode as an API through a tool like Cortex. Without a stop token, the generation will just continue to generate until the token limit is reached, which is not ideal if you know what kind of output you're expecting. This is particularly useful if you're trying to use a few-shot markup like chatML to separate instruction or chat responses.

This change does separate the reverse prompt from interactive mode, so now interactive mode needs to be specified along with reverse prompt for interactive scenarios. The help has been updated accordingly. I could have just added a separate stop token string, but that seemed bloaty.

[SlyEcho]

Reverse prompt will switch to interactive mode right now, but if you pipe in /dev/null to stdin the program will exit. I've used this method to connect main.cpp to Langchain, for example.

[data-angel]

Reverse prompt will switch to interactive mode right now, but if you pipe in /dev/null to stdin the program will exit. I've used this method to connect main.cpp to Langchain, for example.

Thanks for the info - that's good to know! I still like the idea of making it a first-class citizen to align better with the way we typically use other gen models.

[DannyDaemonic]

How does everyone feel about this approach? I think it's cleaner than having a separate set of exit-program prompts that are just a copy and paste of the antiprompt checks. It may require a few scripts to be updated but with 1305 happening, people can just fix both at once.

@data-angel Are you willing to bring this up to date?

[SlyEcho]

I like this more since it is predictable, if I don't enable interactive, it is not enabled for me.

[data-angel]

@DannyDaemonic Thanks for the feedback! It should be up to date now. Let me know if you need anything else.

[data-angel]

You should probably also update common.cpp's gpt_params_parse to reflect the test for interactivity because antiprompts no longer trigger interactive mode. So something like this:

    if (params.prompt_cache_all && 
        (params.interactive || params.interactive_first || params.instruct)) {

Good catch - will update that.

[data-angel]

You should probably also update common.cpp's gpt_params_parse to reflect the test for interactivity because antiprompts no longer trigger interactive mode. So something like this:

    if (params.prompt_cache_all && 
        (params.interactive || params.interactive_first || params.instruct)) {

Good catch - will update that.

This is done.