dspy icon indicating copy to clipboard operation
dspy copied to clipboard

Allow customization of DSPy template

Open wullli opened this issue 9 months ago • 1 comments

Currently the way the instructions and format are passed to the LM uses a hardcoded template with TemplateV2 as far as I understand. I've observed that depending on the model, this format is suboptimal. For example: Llama3-8b-instruct sometimes has difficulty recognizing what are input fields and what are output fields. I think it would be much nicer to have the option to customize all parts of the template by using something like a Jinja template. For example:

{{ instructions }}

{{ part_delimiter }}

Follow the following format.

{%- for field in input_fields %}
 {{ field.prefix }}{{ field.desc }}
{% endfor %}

{%- for field in output_fields %}
 {{ field.prefix }}{{ field.desc }}
{% endfor %}

{{ part_delimiter }}

{%- for field in input_fields %}
 {{ field.prefix }}{{ field.value }}
{% endfor %}

If this is too much work at least dependency injection of the Template class would be cool to have, such that we don't have to fork DSPy for every change in template that we want. I can offer to contribute on this, but I'm not familiar enough with the lib to say how much effort this would be.

wullli avatar May 08 '24 13:05 wullli

could be achieved with a relatively light touch using a cm and context vars?

mikeedjones avatar May 08 '24 16:05 mikeedjones