Experimenting DSPy

[cyyeh]

We wonder how DSPy could help us auto-optimize prompt for us, I've created a branch before: https://github.com/Canner/WrenAI/tree/chore/ai-service/dspy, so please do some experiments on DSPy based on the branch.

DSPy related code is based on wren-ai-service/eval/prompt_optimizer.py, wren-ai-service/eval/dspy_modules and wren-ai-service/eval/optimized.

Please read wren-ai-service/eval/prompt_optimizer.py first so you could better understand the overall workflow

You may use public text-to-sql dataset(for example, spider dataset) to do some experiments, and the final result would be optimized json. We hope we can then retrieve optimized prompt from the json and ingest them into the prompts.

Feel free to discuss with me if you have any other questions, welcome to join our Discord: https://discord.gg/5DvshJqG8Z

[tedyyan]

After talking with Willian, he told me to try this issue first. Thanks, William and Jimmy

[cyyeh]

@tedyyan sure, welcome for contribution! I've assigned you to the issue and let's discuss the issue first in Discord :)

[tedyyan]

see the merge request: https://github.com/Canner/WrenAI/pull/805