llmine_core
llmine_core copied to clipboard
Add strong typed validation for LLM returned values
Background
We have well defined jsonschema
and labels
for extractor prompts for json
and labels
return types respectively.
But currently we don't have any validation after the LLM has responded to an extractor prompt.
New Behaviour
- Check for json type: If returned JSON text from the LLM does not conforms to the given jsonschema use separate LLM call to correct it or retry the prompt.
-
Check for labels type: Returned string must be one of the many values in the
labels_config_json
array. - Text type needs no special validation and retry
Where to make such changes
There is a TODO comment in Line 77 in tasks.py
in run_extracter_chain
method.