guidance
guidance copied to clipboard
microsoft
ongoing string template support
Is your feature request related to a problem? Please describe. the old platform supported string-driven templates.
I'm wondering if you all will be supporting that indefinitely-- mainly because I would like to experiment with having an LLM generate templates to enforce the outputs of a different LLM process.
still possible to do code generation with the new format, but then I need a runtime evaluation (extra safety checks etc)
any thoughts?
I have run into the same problem. I wanted to write a playground for testing guidance. But the use of f-strings means, that the template has to be hardcoded. There is no good way to use a string variable as a template. This severely limits the use of guidance.
Hey! We thought a lot about this, and ultimately came to the conclusion that we could do anything with python syntax we could have before. The one caveat of course is that you can also do a lot more with python, so there are more security considerations.
I think if we were able to create an example that was essentially a safe eval() for guidance code that would solve these use cases. Thoughts? If so, perhaps we could look around for a nice sandbox so we can have a standard way of doing this. (some of these issues will also come up as we set up client/server support for guidance since we will need to ship functions to the server)
A template engine is basically a safe eval(). So yes, something like that would be needed.
Currently I'm using eval('f' + repr(template)), which works, but is not safe.
Hey! We thought a lot about this, and ultimately came to the conclusion that we could do anything with python syntax we could have before. The one caveat of course is that you can also do a lot more with python, so there are more security considerations.
I think if we were able to create an example that was essentially a safe
eval()for guidance code that would solve these use cases. Thoughts? If so, perhaps we could look around for a nice sandbox so we can have a standard way of doing this. (some of these issues will also come up as we set up client/server support for guidance since we will need to ship functions to the server)
Yeah, eval is the direction I'm leaning currently. Would appreciate an example, thanks! I think OpenAI handles sandboxing by using restricted Jupyter Notebooks to execute blocks-- I'm not sure its something guidance needs to take on per se, as long as there were solid examples of safe vs unsafe usage.
A template engine is basically a safe
eval(). So yes, something like that would be needed. Currently I'm usingeval('f' + repr(template)), which works, but is not safe.
Do you think this approach could work with Jinjia2 template? I am creating pretty complex strings and escaping quotes and double quotes becomes very complex to debug very fast, hence with Jinjia2 things would become easier.
