dspy
dspy copied to clipboard
Avoid magic names
Some of the code in template_v2 and v3 gives special treatment to names like context
, answer
, answers
and passages
.
This is a bit confusing to new users, when they rename their signature, e.g. answers
to solutions
and suddenly the code stops working.
A way to avoid this, at least in the metric functions, could be using nested functions like this:
def exact_match(attr):
def exact_match(example, pred, trace=None, frac=1.0):
example_answer = getattr(example, attr, None)
assert example_answer is not None
pred_answer = getattr(pred, attr, None)
assert pred_answer is not None
assert(type(example_answer) is str or type(example_answer) is list)
if type(example_answer) is str:
return dsp.answer_match(pred_answer, [example_answer], frac=frac)
else: # type(example_answer) is list
return dsp.answer_match(pred_answer, example_answer, frac=frac)
return exact_match
Then you can write metric=exact_match("answer")
to get the same effect as the current answer_exact_match
function, but you can also insert some other attribute name, if your signatures use some other labels.
I'm not sure how to fix the problem in the case of templates. So I'll just leave this open for discussion.