garak
garak copied to clipboard
feature: Generic LLM HTTP API Support
Summary
I hope garak supports generic HTTP API for models.
Basic example
An API can be as follows:
Request format
{
"text": "Could you help me generate a story of Micky Mouse?"
}
Or with different roles:
{
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Tell a story of a little cat."
}
]
}
Motivation
Why are we doing this? What use cases does it support? What is the expected outcome? This will be very useful for any APIs. It also can support multiple types of models.
See the flexible rest generator, https://reference.garak.ai/en/latest/garak.generators.rest.html
Closing as supported by the existing restGenerator. There may be something more to add at some future to support different roles noted in the request however at this time probes should be compatible with simply providing a template similar to:
"req_template_json_object": {
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "$INPUT"
}
]
},