Allow dots in $variable names and provide a way to override or replace the ContextVariables class

[Stevenic]

We're exploring integrating SK with the Bot Framework SDK's as a way to leverage Large Language Models (LLMs) for the creation of chat bots. SK seems like a perfect fit but one rough edge is around SK manages memory. The Bot Framework has a very rich state management system that breaks memory down into scopes. These scopes are extensible but the 3 default scopes that are relevant here are:

• conversation - variables are automatically remembered on a per conversation basis so if the bot is being used in 4 different chats there are 4 completely separate sets of variables being remembered.
• user - variables are automatically remembered for each user across all their conversations. So if there are 10 of us in a chat we each have a private set of user variables that are the same across all 4 chats I'm in with the bot.
• temp - variables are transient and only remembered for the turn. This more closely matches the current working memory for SK.
• activity - a pseudo memory scope for referencing properties off the received activity.

To create the best developer experience for our community it would be great if developers could just reference these scoped bags of state directly from the prompt; for example {{$user.name}} could reference the name of the current user, {{$activity.text}} could refence the text of the user massage the bot received, and {{$conversation.workItems}} could reference a list of work items being tracked by the bot.

The alternative is to make the developer manually copy every bot state variable they wish to reference in their prompts to working memory (ContextVariables) before they call into SK. That obviously would work but is a less then ideal developer experience.

To make SK "Bot Framework State Aware" I think 2 features would be needed:

1. We would need to relax the requirement to not include dots in $variable names.
2. We need some way of either replacing the ContextVariables class used by SK (make it an interface and include a clone() method) or let us inherit from and override the property getters and setters...

To go along with this on the Bot Framework side of things I would propose we map flat references like {{$history}} to our temp scope so we would turn that into {{$temp.history}} in our implementation of ContextVariables.

[alexchaomander]

@dluc for visibility

[dluc]

thanks @Stevenic - we're looking into this area while working on increasing the pipeline flexibility. The change should be straightforward, but we just need to make sure it doesn't conflict with the bigger picture. We'll provide updates soon

[evchaki]

@Stevenic, thanks again for the great feedback on this. I made sure we are tracking it with the project team.

[Stevenic]

So we think we have a way on the C# side of things we can optimize passing state from our memory system into SK. We can pre-parse the prompt we want to run into a Block array. We can then look for all the VarBlock instances to get a list of variables needed by the prompt. Then we can copy the required variables from our memory to a ContextVariables instance before running the prompt.

This should work for simple prompt cases but we'd still ideally like the ability to use dots in our variable names. For complex prompts that use semantic function we would we would need the ability to intercept the call to the prompt so we can perform the same variable analysis. We may need to create our own wrapper for defining semantic functions to make that work...

[shawncal]

Good request. @SergeyMenshykh FYI.

[dluc]

I think the work on typescript has taken a break and we haven't heard about similar scenarios elsewhere, so for now I'm closing this. We'll pick up the work once we approach multi-modality and more advanced scenarios dealing with structured data.