dspy icon indicating copy to clipboard operation
dspy copied to clipboard

Empty context causes 4o models to return "N/A"

Open excubo-jg opened this issue 4 months ago • 2 comments

if dspy.Prediction is called with context = [] this causes in chat_adapter.py

def format_fields(fields):
    output = []
    for k, v in fields.items():
        v = _format_field_value(v)
        output.append(f"[[ ## {k} ## ]]\n{v}")

and def format_list(items): if len(items) == 0: return "N/A"

that content in format_turn() content.append(format_fields({k: values.get(k, "Not supplied for this particular example.") for k in field_names})) is set to ['[[ ## context ## ]]\nN/A\n\n[[ ## question ##]]\nMy prompt']

This context makes OpenAI 4o (but not 4 or older versions) reply "N/A" to all prompts. I think the line returning "N/A" should be changed to return "" - could do a pr if this is confirmed.

excubo-jg avatar Oct 07 '24 22:10 excubo-jg