[Bug]: Classification Task: if the LLM produces an output that is a class not specified in the list of labels, we should mark as failure

[rishabh-bhargava]

Describe the bug Currently, an LLM can produce an output that doesn't match one of the labels in labels. In this case, we assume that is the final label produced by the LLM and this leads to a missed example. Instead, we should mark this as a failure, or retry the LLM call.

To Reproduce For the civil-comments dataset, try a prompt like: "Is this comment toxic or not?" in a zero-shot fashion. This should produce some yes or no outputs.

Expected behavior Instead, we should be marking these examples as failures and as a next step, either retrying, or improving the prompt for the user. Earlier, this used to not happen because we would supply the labels list automatically, but now we leave this to the user.