User input and output to the prompt should be sanitized

[sberyozkin]

By default, basic escaping of <, > and a few other characters should be done, but users should also be able to register per/post handlers for sanitizing if they would like to

[geoand]

Do you have examples of where this would be needed?

Also, are you interested in looking into this?

Thanks

[sberyozkin]

Yes, I'm creating issues based on https://developer.nvidia.com/blog/best-practices-for-securing-llm-enabled-applications/ (they reported several CVEs against older versions of the langchain library), which I'm finding as very well prepared, look forward to contributing

[sberyozkin]

Thinking more about it, users can just use Resteasy Reactive filters when necessary, but, some basic escaping may be worth doing by default on in/out, so keeping this issue open for now

[sberyozkin]

I wonder, if parameterizing the plugin input can be more useful, as shown in some demos, which covers the input, sanitizing the LLM output with the API key available seems questionable, just thinking aloud for now.