outlines icon indicating copy to clipboard operation
outlines copied to clipboard

Allow Non-Strict mode in `generate.json`

Open lapp0 opened this issue 1 year ago • 3 comments

Fixes https://github.com/dottxt-ai/outlines/issues/985

Problem

For generate.json(), Sometimes models exhaust their available token length and generate a sequence which isn't valid JSON or doesn't comply with the provided schema.

Solution

Implement strict mode, which defaults to True (same behavior as main)

If generate.json(..., strict=False) is called, an error dict will be returned rather than an error being raised.

{
    'error': 'Unterminated string starting at: line 1 column 13 (char 12)', 
    'error_type': 'JSONDecodeError', 
    'output': '{"this output obviously is not valid JSON',
}

lapp0 avatar Sep 17 '24 20:09 lapp0