bi-att-flow
bi-att-flow copied to clipboard
ForwardEvaluator Bug
In the class "ForwardEvaluator" in the evaluator.py, there is a function: class ForwardEvaluator(Evaluator):
def _get2(context, xi, span):
if len(xi) <= span[0][0]:
return ""
if len(xi[span[0][0]]) <= span[1][1]:
return ""
return get_phrase(context, xi, span)
I think
if len(xi[span[0][0]]) <= span[1][1]:
should be
if len(xi[span[0][0]]) <span[1][1]:
In some cases, len(xi[span[0][0]]) is equal to span[1][1], e.g., the answer is from somewhere to the end of context.
I think you are right. In fact, I will change it to new_span[1][1] = min(len(xi[span[0][0]), span[1][1]) so that it won't output "" just because it extends one more word for some reason. I will let you know if this makes any change to the results. Thanks!