Implement custom clients in addition to ones supported by LangChain
I have customised your tool to add an option to target arbitrary LLM implementations, not only ones supported by LangChains. I have done this to be able to target an LLM exposed via a custom REST API.
To use this mode user has to provide a custom python module implementing two methods. Please see ps_fuzz/custom.py for sample implementation (it uses a local transformer-based model for testing).
To test: prompt-security-fuzzer --target-provider custom --target-model ps_fuzz.custom -b
If you find it useful and willing to merge in -- please let me know. I will adjust README and provide sample client for REST API.
Hi @abbbe
It seems I am not able to add PR to your cloned repo (it doesn't allow anonymous contributions).
So please, update your PR: add this line to setup.py in the install_requires=[] list (after "fastparquet" line):
+ "setuptools>=80.0.0,<81.0.0"
This is lacking for your addition of the new "plugin-based" system.
Also, if you can, please add a short section in README describing your new "plugins" feature.
Also, suggest you pull the latest commits from the upstream repo (https://github.com/prompt-security ps-fuzz), it contains some bugfixes.