unitxt icon indicating copy to clipboard operation
unitxt copied to clipboard

Add type checking for prediction_type of complex tasks like RAG

Open yoavkatz opened this issue 1 year ago • 0 comments

In complex tasks, like RAG, there prediction_type is a Dictionary with a fixed fields.

Today, it is modeled as

prediction_type: "Dict"

We want to be able to write:

“prediction_type”: “Dict[{“answer”: “str”, “contexts” : “List[str]“, “contexts_id”: “List[str]“, “is_answerable”: “Boolean”]"

yoavkatz avatar Jun 24 '24 14:06 yoavkatz