ps-fuzz icon indicating copy to clipboard operation
ps-fuzz copied to clipboard

Implement custom clients in addition to ones supported by LangChain

Open abbbe opened this issue 8 months ago • 1 comments

I have customised your tool to add an option to target arbitrary LLM implementations, not only ones supported by LangChains. I have done this to be able to target an LLM exposed via a custom REST API.

To use this mode user has to provide a custom python module implementing two methods. Please see ps_fuzz/custom.py for sample implementation (it uses a local transformer-based model for testing).

To test: prompt-security-fuzzer --target-provider custom --target-model ps_fuzz.custom -b

If you find it useful and willing to merge in -- please let me know. I will adjust README and provide sample client for REST API.

abbbe avatar Jun 10 '25 18:06 abbbe

Hi @abbbe

It seems I am not able to add PR to your cloned repo (it doesn't allow anonymous contributions).

So please, update your PR: add this line to setup.py in the install_requires=[] list (after "fastparquet" line):

+        "setuptools>=80.0.0,<81.0.0"

This is lacking for your addition of the new "plugin-based" system.

Also, if you can, please add a short section in README describing your new "plugins" feature.

Also, suggest you pull the latest commits from the upstream repo (https://github.com/prompt-security ps-fuzz), it contains some bugfixes.

vitaly-ps avatar Jun 17 '25 17:06 vitaly-ps