Improve Planner to Support invalid ASK - Issue 159

[rogerbarreto]

Motivation and Context

Resolve Related Issue: Improve Planner prompt to support invalid ASK

This change mitigate the potential of generated plans with non existing Skills / Functions not provided.

If you ask for a Plan with insufficient Skills to fulfill, it will fail with a reason why. context.ErrorDescription

Description

This change makes an additional query to the LLM providing only the available skills and the Goal without mixing it with the plan rules.

This provides clarity when asking for plans that cannot be created and avoid the potential of LLM wrong assumptions, hallucinations using the few shots on the planning creation prompt.

This changes adds also a post verification when the plan is created to ensure the functions returned by the plan are indeed present in the SkillCollection.

Contribution Checklist

• [x] The code builds clean without any errors or warnings
• [x] The PR follows SK Contribution Guidelines (https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
• [x] The code follows the .NET coding conventions (https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions) verified with dotnet format
• [x] All unit tests pass, and I have added new tests where possible
• [x] I didn't break anyone :smile:

[MovGP0]

See also #523 which wants to remove the planner altogether

[lemillermicrosoft]

See also #523 which wants to remove the planner altogether

PlannerSkill - not Planner though :) But yeah, we will still want to do something to address this issue. Did not do in that PR to reduce scope of changes.

[shawncal]

@RogerBarreto I think we can close this draft. Can you confirm?