kcaverly

I believe the LiteLLM backend should interface with gptv, but I imagine we will need to accomodate for this with a distinct backend, so more work would be required, but...

Sorry this should be merging to the backend refactor branch.

This is likely going to be possible on a Language Model specific basis. For example, LiteLLM makes costing super simple, and this would be very simple to implement. With this...

If you're keen you are welcome to take a look at the new backends infra [here](https://github.com/stanfordnlp/dspy/tree/backend-refactor/dspy/backends). We've been working on it in the backend-refactor branch. Currently we've just got LiteLLM...

We should have the flexibility to handle any LM that implements the BaseLM class. As we can’t guarantee that every LM will offer costing on a per request basis, maybe...

Just catching up on this. It may be helpful for folks to take a look at the new [Template](https://github.com/stanfordnlp/dspy/blob/backend-refactor/dspy/primitives/template.py) class, it should contain all the TemplateV2/TemplateV3 functionality. Additionally, all functionality...

FWIW - the new backed system would allow you to provide your own templates and supports chat mode. If you have a fuller example you can share, I would be...

Ive been working on it here: #717. So far, im passing it during generation. The backend has a default template argument, that can be overridden in modules when the backend...

If you take a look at the JSONBackend, we do something very similar. For json mode models, we prompt the model to complete the json as an incomplete object, as...

Everything is currently stuffed into on message, but instead of providing the question and asking for a json response, we send an incomplete json object and ask to complete it....