haystack
haystack copied to clipboard
`Label` skips `answer` validation if `no_answer=None`
Describe the bug
If the default value of no_answer is used the validation of no_answer here is skipped. Haystack then sets the default here which means that if the Label object is e.g. serialized and deserialized, the validation will kick in and potentially raise.
Error message Error that was thrown (if available)
Expected behavior
The code should be like this so that the validation of no_answer, answer.context and answer.answer is happening correctly.
if self.answer is not None:
if no_answer is None:
# Set default before validation
no_answer = self.answer.answer == "" or self.answer.answer is None
if no_answer == True:
if self.answer.answer != "" or self.answer.context:
raise ValueError(f"Got no_answer == True while there seems to be an possible Answer: {self.answer}")
elif no_answer == False:
if self.answer.answer == "":
raise ValueError(
f"Got no_answer == False while there seems to be no possible Answer: {self.answer}"
)
Additional context Add any other context about the problem here, like document types / preprocessing steps / settings of reader etc.
To Reproduce
from haystack.schema import Label, Answer, Document
# doesn't raise
label = Label(answer=Answer(answer="", context="some value"), query="test", document=Document("dasd"), is_correct_answer=True, is_correct_document=True, origin="user-feedback")
# raises but shouldn't
Label.from_dict(label.to_dict())
FAQ Check
- [ ] Have you had a look at our new FAQ page?
System:
- OS:
- GPU/CPU:
- Haystack version (commit or version number):
- DocumentStore:
- Reader:
- Retriever: