outlines
outlines copied to clipboard
Allow Non-Strict mode in `generate.json`
Fixes https://github.com/dottxt-ai/outlines/issues/985
Problem
For generate.json(), Sometimes models exhaust their available token length and generate a sequence which isn't valid JSON or doesn't comply with the provided schema.
Solution
Implement strict mode, which defaults to True (same behavior as main)
If generate.json(..., strict=False) is called, an error dict will be returned rather than an error being raised.
{
'error': 'Unterminated string starting at: line 1 column 13 (char 12)',
'error_type': 'JSONDecodeError',
'output': '{"this output obviously is not valid JSON',
}