semantic-kernel
                                
                                 semantic-kernel copied to clipboard
                                
                                    semantic-kernel copied to clipboard
                            
                            
                            
                        Hooks for logging/replay of skills.
Since skills can be built out of other skills, I would like to get detailed logging information on what skills get called in the process of running a skill and what values it produced both for understanding what happened and for replay for unit testing skills. For example, given a skill calling out to another skill expensiveComputation like
The answer is {{expensiveComputation}}. What is the question?
I would like to have a way to get a log showing the value return by the native skill expensiveComputation (the actual prompt is given to the ILogger although it would also be nice to have a structured way to get it). Something like
{
    "skills": {
        "expensiveComputation": "42"
    },
    "prompt": "The answer is 42. What is the question?"
}
Although if expensiveComputation were a semantic skill with its own nested computations, then maybe there could be further nesting in the log. Furthermore, once this log existed it could be used to mock those skills so the computation wouldn't have to be redone when wanting to test other parts of the system. In addition to avoiding expensive computation, this could be particularly useful if those skills require filesystem or other access that isn't available when testing.
I do not think the specific log format is appropriate to this project. Instead, I was looking for the lowest impact way to adjust the APIs to allow for this functionality to be built in client code. It looks like PromptTemplateEngine is where the relevant work is done, but simply putting in my own version without modifying this project doesn't work as it relies on internal members of Block to perform its computations. One option would be to make those members public, but in the interest of limiting new externally visible APIs, I've made a branch which adds virtual methods to PromptTemplateEngine at the point where it computes the values of the blocks which could be used to either record the values computed or overriding the computation with a different value read from a log (and modifies KernelBuilder to make it easier to drop in a different PromptTemplateEngine implementation). These extension points could probably be used for other uses I haven't thought of as well.
If this solution is acceptable, I can make a PR, but the contributing guidelines requested I make a feature request to discuss first, and I'm not sure my solution is actually the best way to handle this, anyway.