microsoft
Allow SequentialPlanner to use LLM to ensure formatting and chaining of functions
For the needs of a prototype, I had to chain actions (and so functions). Currently, SequentialPlanner is able to chain functions, but output of a function is directly linked to the parameter of a following function. Considering users, of the Copilot ChatApp for example, that can ask for any kind of request with supported functions/skills, we don't know how functions will be chained. A common issue could be that one function return a whole text with, let's say the properties of an email, and the final function/action will be to send an email based on the properties. The last function/action will ask for an email as parameter but we only return a whole text/set of properties from the initial function. Allowing SequentialPlanner to use LLM to prepare properties before the execution of a function if it contains variables from a previous step will allow an easier chaining of function and more use cases/action plan to be handled without any customization and less complexity in skills implementation. I have made this implementation for my prototype, if it make sense, I can submit a PR with my changes.
but output of a function is directly linked to the parameter of a following function.
This is the default behavior. But there is Outputs on the Plan object which can be populated to specify a named output to be used elsewhere in the plan. SequentialPlanParser adds to this using setContextVariable.
However, allowing a function to define it's output and schema is something that will go a long way here and needs to be designed still. Work in progress with the team.
Thanks for the issue and PR, will spend more time reviewing. My initial response though is that the Plan object should not have that tight of a coupling with a semantic function as it's meant to be a well defined plan that can be reliably executed.
Could maybe also work with a parameter in the plan which can be specifically set to true if inputs of the function needs specific format ? My through is that if we go with plans which only link functions with specific outputs/inputs, we will probably finish with skills that can only be used to be linked with others. With the approach I tested, it allows me to have at the same time functions which can be used alone or inside a plan. The other way could also be inside each skill/function to ask LLM to format received inputs according to required formats and user intent (less generical but more controlable).
We're designing some improvements to how outputs/inputs are handled in general to enable a lot more flexibility.
The other way could also be inside each skill/function to ask LLM to format received inputs according to required formats and user intent (less generical but more controlable).
Additionally, if you include the right set of skills (maybe we need to add some of these to CoreSkills), plans can be generated automatically. Imagine a JsonSkill.Select that takes a string input (could be of any format, does not need to be json) and selects a set of properties from the input. If it's json-like, LLM can do it well. If it's not, LLM in some situations, can also do very well.
An example can be seen here: https://devblogs.microsoft.com/semantic-kernel/plugging-into-jira-from-semantic-kernel/
It is a very interesting skill, I was not aware about this one. And yes, integrating it to CoreSkills would probably be a must-have. I am wondering if this approach works well if the initial task return not a JSON but a text (including maybe a part of JSON). I am in this situation.
Also, it is an interesting conversation to understand if this kind of behavior must be a skill or part of the framework.
I will my situation with this approach to see what is the behavior.
Closing stale issues. Please re-open or start a new issue if you are still having issues.
Re-opening to make sure these pieces are tracked:
- Potential core plugin or semantic function for transforming inputs/outputs
- Addition of
rationaleto sequential planner and Plan object.