TextMemorySkill's Recall function blocks threads and causes WPF app to freeze

[SenseYang]

Describe the bug I crafted a WPF app and hooked a button with a RelayCommand that will eventually invoke ISKFunction.InvokeAsync method. Even without the await, it will still blocks the WPF app. And this exact same function, when running with a console app, produced results and finished within several seconds. I am using <PackageReference Include="Microsoft.SemanticKernel" Version="0.12.207.1-preview" /> To Reproduce Steps to reproduce the behavior:

1. Create a WPF app and add a button to the view, and create a new ISKFunction by calling IKernel.CreateSemanticFunction.
2. Configure the button with a RelayCommand.
3. Let the RelayCommand to invoke ISKFunction.InvokeAsync
4. Run debug in Visual Studio.

Expected behavior Clicking the button shouldn't block the WPF app.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Windows 11
• IDE: Visual Studio
• NuGet Package Version: <PackageReference Include="Microsoft.SemanticKernel" Version="0.12.207.1-preview" />

Additional context The VS debug output also shows some interesting information:

The thread 0x5fb8 has exited with code 0 (0x0).
The thread 0x25f0 has exited with code 0 (0x0).
Exception thrown: 'System.IO.IOException' in System.Net.Sockets.dll
Exception thrown: 'System.IO.IOException' in System.Net.Security.dll
Exception thrown: 'System.IO.IOException' in System.Private.CoreLib.dll
The thread 0x622c has exited with code 0 (0x0).
The thread 0x7b14 has exited with code 0 (0x0).
The thread 0x3868 has exited with code 0 (0x0).
The thread 0x1788 has exited with code 0 (0x0).

But the WPF process is just blocked without seeing any exceptions at all.

[runceel]

I have created a plain WPF app that invokes ISKFunction (a semantic function) on .NET 7. Plain means without any packages. Semantic Kernel version is 0.13.442.1-preview.

The source code is:

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.SkillDefinition;
using System;
using System.Windows;

namespace WpfApp2;
/// <summary>
/// Interaction logic for MainWindow.xaml
/// </summary>
public partial class MainWindow : Window
{
    private readonly IKernel _kernel;
    private readonly ISKFunction _function;
    public MainWindow()
    {
        InitializeComponent();

        _kernel = Kernel.Builder
            .Configure(config =>
            {
                config.AddAzureChatCompletionService(
                    "your deployment model name",
                    "https://your resource name.openai.azure.com/",
                    "your api key");
            })
            .Build();

        _function = _kernel.CreateSemanticFunction("""
            Please generate {{$input}} interesting words.
            """);
    }

    private async void Button_Click(object sender, RoutedEventArgs e)
    {
        var number = Random.Shared.Next(10).ToString();
        textBlock.Text = $"{DateTimeOffset.UtcNow}, Input: {number}: \n{await _function.InvokeAsync(number)}";
    }
}

<Window x:Class="WpfApp2.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        mc:Ignorable="d"
        Title="MainWindow" Height="450" Width="800">
    <StackPanel>
        <Button Content="Call the function" Click="Button_Click" />
        <TextBlock x:Name="textBlock" />
    </StackPanel>
</Window>

I did not encounter any issues, so I assume this is regarding the RelayCommand.

[SenseYang]

Thanks. I modified my code to create a button click handler like what you did, instead of using the RelayCommand, but the app still got frozen at debugging run in VS.

One thing needs to be mentioned though, my Kernel imported the TextMemorySkill, and the prompt of the inline function I used is like this:

const string skPrompt = @"
            ChatBot can have a conversation with you about requirements and guidelines to implement a policy, and an actual policy.
            It can give explicit answers or say 'I don't know' if it does not have an answer.

            Here's the requirements for a policy:
            - {{$controlQuestion}} {{recall $controlQuestion}}
            
            Here's information about an actual policy:
            - {{$policyQuestion}} {{recall $policyQuestion}}

            Chat:
            {{$history}}
            User: {{$userInput}}
            ChatBot: ";

        var chatFunction = _kernel.CreateSemanticFunction(skPrompt, maxTokens: 3000, temperature: 0.8);
        return chatFunction;

It uses the recall function inside the function. And if I don't use the TextMemorySkill and remove the recall function usage, everything works fine now.

I think the TextMemorySkill and recall function may have some low level implementation that is not compatible to (at least) WPF's thread management.

[runceel]

The recall skill looks blocking a thread. The following code is recall function. It calls ToEnumerable method to get result.

https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel/CoreSkills/TextMemorySkill.cs#L124

I thought it might be a good idea to change the title of the Issue to something like TextMemorySkill's Recall function blocks threads to clarify this issue.

[dluc]

the issue should be solved https://github.com/microsoft/semantic-kernel/pull/987

[SenseYang]

the issue should be solved #987

Thanks, will verify.

When will the next NuGet package version be released? Thanks. @dluc

[runceel]

@dluc I guess the following is prefer for single result case:

    /// <summary>
    /// Semantic search and return up to N memories related to the input text
    /// </summary>
    /// <example>
    /// SKContext["input"] = "what is the capital of France?"
    /// {{memory.recall $input }} => "Paris"
    /// </example>
    /// <param name="text">The input text to find related memories for</param>
    /// <param name="context">Contains the 'collection' to search for the topic and 'relevance' score</param>
    [SKFunction("Semantic search and return up to N memories related to the input text")]
    [SKFunctionName("Recall")]
    [SKFunctionInput(Description = "The input text to find related memories for")]
    [SKFunctionContextParameter(Name = CollectionParam, Description = "Memories collection to search", DefaultValue = DefaultCollection)]
    [SKFunctionContextParameter(Name = RelevanceParam, Description = "The relevance score, from 0.0 to 1.0, where 1.0 means perfect match",
        DefaultValue = DefaultRelevance)]
    [SKFunctionContextParameter(Name = LimitParam, Description = "The maximum number of relevant memories to recall", DefaultValue = DefaultLimit)]
    public async Task<string> RecallAsync(string text, SKContext context)
    {
        var collection = context.Variables.ContainsKey(CollectionParam) ? context[CollectionParam] : this._collection;
        Verify.NotNullOrWhiteSpace(collection, $"{nameof(context)}.{nameof(context.Variables)}[{CollectionParam}]");

        var relevance = context.Variables.ContainsKey(RelevanceParam) ? context[RelevanceParam] : this._relevance;
        if (string.IsNullOrWhiteSpace(relevance)) { relevance = DefaultRelevance; }

        var limit = context.Variables.ContainsKey(LimitParam) ? context[LimitParam] : this._limit;
        if (string.IsNullOrWhiteSpace(limit)) { limit = DefaultLimit; }

        context.Log.LogTrace("Searching memories in collection '{0}', relevance '{1}'", collection, relevance);

        // TODO: support locales, e.g. "0.7" and "0,7" must both work
        var limitInt = int.Parse(limit, CultureInfo.InvariantCulture);
        var relevanceThreshold = float.Parse(relevance, CultureInfo.InvariantCulture);

        // Search query
        var memories = context.Memory
            .SearchAsync(collection, text, limitInt, relevanceThreshold, cancellationToken: context.CancellationToken);
        if (limitInt == 1)
        {
            var memory = await memories.FirstOrDefaultAsync(context.CancellationToken).ConfigureAwait(false);
            return memory?.Metadata.Text ?? string.Empty;
        }

        var all = await memories.ToListAsync(context.CancellationToken).ConfigureAwait(false);
        if (all.Count == 0)
        {
            context.Log.LogWarning("Memories not found in collection: {0}", collection);
            return string.Empty;
        }

        return JsonSerializer.Serialize(memories.Select(x => x.Metadata.Text));
    }

[lemillermicrosoft]

Closing issue, please re-open if you have issues verifying @SenseYang