dspy icon indicating copy to clipboard operation
dspy copied to clipboard

Allow to set LM and RM per module on constructors and forward func

Open neoxelox opened this issue 1 year ago • 2 comments

This PR allows to set LM and RM per module on their constructors or dynamically when calling them.

Note that I removed the query_only=True parameter on the existent with dsp.settings.context in the predictor forward because I checked and that parameter is only used on dsp.TemplateV2 but dspy is using dsp.TemplateV3.

neoxelox avatar Feb 14 '24 15:02 neoxelox

Thank you so much @Neoxelox !

btw @KarelDO is this what you have in mind too?

okhat avatar Feb 14 '24 15:02 okhat

Pinging @KarelDO also to check if this is how he'd do it

okhat avatar Feb 17 '24 05:02 okhat

I'm closing this PR because I have seen that a lot of places (outside of the predictor) the code is relying on the global set dsp.settings.lm, ex.: ChainOfThought / ChainOfThoughtWithHint / compile() / majority_vote() / bootstrap / the history itself...

So I'm not feeling confident enough to make that change myself and risking library stability. I will wait until the big refactor is done, maybe you even add this feature yourselves.

neoxelox avatar Feb 18 '24 10:02 neoxelox