semantic-kernel icon indicating copy to clipboard operation
semantic-kernel copied to clipboard

SequentialPlanner does not seem to be using TextEmbeddings

Open MikeYeager opened this issue 2 years ago • 0 comments

Describe the bug SequentialPlanner is not using the in memory TextEmbeddings of the skills being loaded.

To Reproduce Steps to reproduce the behavior:

  1. I was following this blog post (from all of 6 weeks ago), but it seems to be out of date already.
  2. I created a console app with the following code. If I remove the .AddAzureTextEmbeddingGenerationService(), kernel.UseMemory() and SequentialPlannerConfig, it works as expected, but without using embeddings to create the plan.
  3.     var kernel = KernelBuilder.Create();
         kernel.Config
             .AddAzureTextCompletionService(
                 deploymentName: "text-davinci-003",
                 endpoint: endpoint,
                 apiKey: apikey)
             .AddAzureTextEmbeddingGenerationService(
                 deploymentName: "text-embedding-ada-002",
                 endpoint: endpoint,
                 apiKey: apikey);
    
         kernel.UseMemory(new VolatileMemoryStore());
    
         //Load up the skills the planner will use
         var skillsDirectory = Path.Combine(Directory.GetCurrentDirectory(), "skills");
         kernel.ImportSemanticSkillFromDirectory(skillsDirectory, "SummarizeSkill");
         kernel.ImportSemanticSkillFromDirectory(skillsDirectory, "WriterSkill");
    
         //Define a skill on the fly
         string skPrompt = """
             {{$input}}
    
             Rewrite the above in the style of Shakespeare.
             """;
         kernel.CreateSemanticFunction(skPrompt, "shakespeare", "ShakespeareSkill", maxTokens: 2000, temperature: 0.2, topP: 0.5);
    
         var goal = "Tomorrow is Valentine's day. I need to come up with a few date ideas.\r\nShe likes Shakespeare so write using his style. E-mail these ideas to my significant other.";
         var plannerConfig = new SequentialPlannerConfig { RelevancyThreshold = 0.78 };
         var planner = new SequentialPlanner(kernel, plannerConfig);
         var plan = await planner.CreatePlanAsync(goal);
         var results = await kernel.RunAsync(plan);
         Console.WriteLine(results);
    

Expected behavior The SequentialPlanner native skill is quite different than the planner skill used in the article, but it seems to me that SequentialPlanner should use the embeddings for the loaded skills to create the plan. I can see the embeddings loaded into the memory store, but the plan that is created does not use any of the skills stored in the memory store and does not generate any output.

Desktop (please complete the following information):

  • OS: Windows
  • IDE: Visual Studio
  • NuGet Package Version: 0.14.547.1
  • SequentialPlanner Version: 0.14.547.1

MikeYeager avatar May 16 '23 22:05 MikeYeager