dspy
dspy copied to clipboard
Shadowing signature fields causes weird errors
I was just defining a signature like this:
class Signature(dspy.Signature):
__doc__ = textwrap.dedent(
"""I gave my language model a task, but it failed. Figure out what went wrong,
and write instructions to help it avoid the error next time.""",
)
task_description: str = dspy.InputField(desc="What I asked the model to do")
model_output: str = dspy.InputField(desc="The output of the model")
error: str = dspy.InputField(desc="The validation error trigged by the models output")
explanation: str = dspy.OutputField(desc="Explain what the model did wrong")
instructions: str = dspy.OutputField(desc="Instructions for the model to do better next time")
but this fails with
raise TypeError(
TypeError: Field 'instructions' in 'Signature' must be declared with InputField or OutputField. field.json_schema_extra=None
Why? Because instructions
is a "reserved word" of dspy.Signature
.
There are a number of other reserved words, like fields
, insert
, prepend
, append
, equals
.
At least a few of those have already caused me trouble in the past.
On top of that there are "pydantic reserved words", like schema
that I've also had issues with.
Pydantic 2.x is trying to clean this up, by moving all their special words to the prefix model_
, such as model_schema
instead of schema
. Just now I made a field with name model_output
and got this warning from pydantic: "...pydantic/_internal/_fields.py:149: UserWarning: Field "model_output" has conflict with protected namespace "model_"
I think we should similarly move all the dspy related words to some prefix, like dspy_instructions
, dspy_equals
etc.
This causes only a small inconvenience for dspy backend developers (e.g. you are making a new optimizer), but make life a lot easier for end users (that is, people just defining a casual signature.)
On top of that, we should make sure to throw informative errors in the cases where there's still a clash. Like if the user uses model_schema
or dspy_instructions
as a field name for some reason.