Using Sematic kernel in Bot Framework SDK for great conversations

[sagarsat]

Use case: How can we use sematic kernel to detect user intent and route to proper API function/skill dynamically. For e.g. For user query "I need information on my purchase order" -> it should route to PO skill and then ask user for PO number and then call the PO API

versus

For user query "I need to know my warranty status for serial no XYZ" -> it should route to Warranty Skill and then call the warranty API

Any ideas around this? Thanks

[sagarsat]

tagging for more reach. @microsoftShannon @alexchaomander

[evchaki]

@sagarsat , thanks for the idea. It sounds like you would like to see Semantic Kernel being available to use in the Bot Framework SDK - is that correct? So that, for your example above the planner would find the intent and call a Plugin?

[sagarsat]

@evchaki Yes, I was looking for an example in samples above where the planner would find the intent and call a Plugin. Can you route me to that sample if its there?

[sagarsat]

@evchaki any update on above request. I'm looking for a basic implementation with SK where the SK can understand the intent and route to Plugin/Skill dynamically.

[sagarsat]

Hi @evchaki any updates?

@microsoftShannon @alexchaomander adding for better reach.

[rameshneudesic]

HI @evchaki,

I also have similar requirements for my application. Appreciate your help/guideline to get the solution.

[evchaki]

@rameshneudesic and @sagarsat - thanks for the ping on this. We are working with the Bot Framework team and talking about this use case. We will let you know once we define the details on how this might work.

[sagarsat]

@evchaki Thanks for the update. I will keep checking this thread for an update.

[guru98]

using bot framework you have different way to do that and you have to figure out which is the best mix bw sk and, for example dialogs and step by step flows. I'm working for a while with bot framework and i thing that sk could replace lu as is more strong and flexible so we could implement an ad hoc recognize. When the recognizer identify the intent you have to deal with the parametes for that intent and if not all are available ask for them. I thing that the bot framework can handle this part better than sk but after that you can use bot tools or sk at your convinience. Anyway best practice for the integration could be very usefull and this, in my mind, is a very pratical scenarious to use sk

[raffi1965]

I was struggling last few days from thoughts of how to integrate SK and bot framework. Now its clear that intent detection from SK is not an option to run SK skill. Will be waiting for some updates

[SOE-YoungS]

I am actively using SK in a Bot Framework SDK Teams bot in an enterprise environment.

In the bot controller, we pull in the Kernel via dependency injection.

In "OnMessageRecievedAsync", we then run the "TurnContext.Activity.Text" through the SK intent detection skill & / or check for a !Command at the beginning of the text.

Further processing the text from there is as simple as writing any other SK / Bot Framework code.

Note: we are not using Dialogs for this, as we wanted more fine grained control over the processing of prompts, however the principal is the same. We do however use a Waterfall dialog, to capture an access token (for the current user) for later use in API calls from the bot to other services (IE: MS Graph).

In our bot, LU is not used at all :)

[alliscode]

We're working on a proof of concept for integrating Bot Framework with Semantic-Kernel. We'll update with details once we have something.

[mahi83]

Any updates on this?

[rvinothrajendran]

Could this plug-in be of assistance to you? You can find it at this link: AzureAI.Community.Microsoft.Semantic.Kernel.PlugIn.Intent

[mahi83]

Could this plug-in be of assistance to you?

You can find it at this link: AzureAI.Community.Microsoft.Semantic.Kernel.PlugIn.Intent

This looks more like an intent recognition use case. I'm looking for something that would be able to identify the set of skills to use, and complete them. Do you think this would be able to do that?

[rvinothrajendran]

Could this plug-in be of assistance to you? You can find it at this link: AzureAI.Community.Microsoft.Semantic.Kernel.PlugIn.Intent

This looks more like an intent recognition use case. I'm looking for something that would be able to identify the set of skills to use, and complete them. Do you think this would be able to do that?

Could you kindly provide an example?

[mahi83]

This is just an example: Message: I am looking to find a trip to Europe that cost within $1000 next month.

Skills:

• Flight finder: A tool to find all flights to Europe
• Hotel finder: A tool to find all hotels in Europe
• Car finder: A tool to find costs associated with cars
• Calculator: A tool to add costs
• Insurance: A tools to find all international insurance
• Visa finder: A tool to detect if you need visa for the country visiting

So in this example, I want SK to identify the skills we need are flight finder, hotel finder, car finder, calculator (using any of the planner). Then I want to apply these skills/api calls/functions calls and come up with an answer.

Will that plugin work for this example?

[rvinothrajendran]

In my understanding, yes, you can accomplish that, but it won't be a single step. You'll need to follow a step-by-step process, such as first finding the location and then exploring travel options like flights, hotels, cars, and so on.

[mahi83]

In my understanding, yes, you can accomplish that, but it won't be a single step. You'll need to follow a step-by-step process, such as first finding the location and then exploring travel options like flights, hotels, cars, and so on.

Could you elaborate a little more on what you mean by step by step process? Does the bot framework extension shared above return a list of all steps, and we apply them step by step?

[martinlarosa]

@mahi83 check this out: https://learn.microsoft.com/en-us/semantic-kernel/ai-orchestration/planners/

[mahi83]

@mahi83 check this out: https://learn.microsoft.com/en-us/semantic-kernel/ai-orchestration/planners/

Hmmmm. I'm trying to ask is there a formalized way of integrating this into bot framework? When using promptflow, semantic kernel integrates with it very nicely.

[rvinothrajendran]

@mahi83 Can you accomplish this concept in SK without using the Bot Framework?

[mahi83]

@mahi83 Can you accomplish this concept in SK without using the Bot Framework?

I believe so using what @martinlarosa shared. So one way to integrate bot framework is first to create an api that does it and use bot framework to connect to it. Was wondering if you all had any other OOB approaches/plugins.

[rvinothrajendran]

@mahi83 Can you accomplish this concept in SK without using the Bot Framework?

I believe so using what @martinlarosa shared. So one way to integrate bot framework is first to create an api that does it and use bot framework to connect to it. Was wondering if you all had any other OOB approaches/plugins.

I currently don't think there are any other approaches available at the moment

[martinlarosa]

Oh ok, then yes, you'll need to wait until they release the sample.

[markwallace-microsoft]

All .Net issues prior to 1-Dec-2023 are being closed. Please re-open, if this issue is still relevant to the .Net Semantic Kernel 1.x release. In the future all issues that are inactive for more than 90 days will be labelled as 'stale' and closed 14 days later.