guardrails
guardrails copied to clipboard
[feat] Improve LLM output parsing to discover JSON not within a code block
Description Add another step to our parsing logic that checks for JSON not within a code block
Why is this needed Some open source models don't escape JSON in a code block. i.e They'll just return:
{ "a": 1 }
instead of
```json
{ "a": 1 }
```
Implementation details
See existing parsing logic here.
It might be able to be extended to support both code blocks and {}
or []
blocks, or similar but new functions could be written and added to the top level parsing here
End result All three of the following LLM outputs should be parseable to extract the JSON within them:
- JSON code block
Here is the data you requested
```json
{ "a": 1 }
```
- Generic code block
Here is the data you requested
```
{ "a": 1 }
```
- JSON outside of a code block
Here is the data you requested
{ "a": 1 }
Bonus points for supporting JSON Arrays as well:
Here is the data you requested
[ { "a": 1 }]
JSON objects were addressed in #548.
JSON arrays still need to be handled; will leave open for that.
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 14 days.
This issue was closed because it has been stalled for 14 days with no activity.