Python: implement Hooks

[eavanvalkenburg]

Motivation and Context

This pull request includes significant changes across multiple files, mainly related to the addition of hooks and the modification of function invocations in the semantic_kernel module. The changes also include the addition of a new sample and a YAML file, and modifications to the __init__.py files.

Addition of Hooks:

• python/semantic_kernel/hooks/__init__.py: Added new context classes, protocol classes, helpers (utils and const) and a hook filter decorator.

Other Additions and Modifications:

• python/samples/kernel-syntax-examples/azure_chat_gpt_api_with_hooks.py: Added a new Python script that demonstrates the usage of hooks with the Azure Chat GPT API.

• python/semantic_kernel/connectors/ai/open_ai/utils.py: Modified the _describe_tool_call and parse_param functions to handle more complex parameter types. [1] [2]

• python/semantic_kernel/functions/kernel_function.py: Modified the invoke and invoke_stream methods to include pre and post hooks. [1] [2] [3] [4]

• python/semantic_kernel/functions/kernel_function_from_prompt.py: Modified the _invoke_internal and _invoke_internal_stream methods to include pre and post prompt render hooks. [1] [2]

Removals:

• python/semantic_kernel/events: Removed the previously used events.

New Exceptions:

• python/semantic_kernel/exceptions/kernel_exceptions.py: Added new exception classes OperationCancelledException and HookInvalidSignatureError. [1] [2]

Changes to Function Decorator:

• python/semantic_kernel/functions/kernel_function_decorator.py: Modified the _parse_internal_annotation function to handle list and dict types.

Fixes: #3038

Contribution Checklist

• [x] The code builds clean without any errors or warnings
• [x] The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• [x] All unit tests pass, and I have added new tests where possible
• [ ] I didn't break anyone :smile:

[markwallace-microsoft]

Python 3.8 Test Coverage Report •

File	Stmts	Miss	Cover	Missing
semantic_kernel
kernel.py	261	28	89%	122, 124, 182–185, 187–191, 193–196, 203, 210–219, 254–257, 267–272, 277, 281–284, 466–467
semantic_kernel/connectors/ai/open_ai
utils.py	46	21	54%	27–28, 51–55, 62–63, 104, 142, 144, 146, 148, 150, 155–162
semantic_kernel/events
function_invoking_event_args.py	6	1	83%	35
kernel_events_args.py	14	3	79%	38, 41–42
semantic_kernel/functions
function_result.py	30	9	70%	45, 47–48, 58–63
kernel_function.py	72	2	97%	141, 160
kernel_function_decorator.py	67	1	99%	101
kernel_function_from_prompt.py	162	4	98%	170, 198, 299, 369
TOTAL	5545	1010	82%

Python 3.8 Unit Test Overview

Tests	Skipped	Failures	Errors	Time
1211	11 :zzz:	0 :x:	0 :fire:	20.899s :stopwatch:

[bashimr]

@eavanvalkenburg @moonbox3. I have a general feedback on the hooks implementation. From my point of view as an LLM developer, the most common use case for this is to be able to get the token usage after invoking a semantic function or executing an agent (with multiple tools invocations).

I feel that the mechanism mentioned in the example is somehwat complicated, specially for the inexperienced python developers. I would recommend adopting the with-statement context as it is done in Langchain. So, getting the tokens after an LLM call is as easy as this:

        with get_openai_callback() as callback:
            response = agent.invoke({"input": "What is 2+2"})
            print(f"Prompt Tokens: {callback.prompt_tokens}")
            print(f"Completion Tokens: {callback.completion_tokens}")
            print(f"Total Tokens: {callback.total_tokens}")

This allows new developers to use the 'with' statement and default callback implementation for common use cases like getting tokens but more experienced developers can implement their custom callback functions to do more sophisticated stuff.

[eavanvalkenburg]

@eavanvalkenburg @moonbox3. I have a general feedback on the hooks implementation. From my point of view as an LLM developer, the most common use case for this is to be able to get the token usage after invoking a semantic function or executing an agent (with multiple tools invocations).

I feel that the mechanism mentioned in the example is unnecessarily complicated, specially for the inexperienced python developers. I would recommend adopting the with-statement context as it is done in Langchain. So, getting the tokens after an LLM call is as easy as this:

        with get_openai_callback() as callback:
            response = agent.invoke({"input": "What is 2+2"})
            print(f"Prompt Tokens: {callback.prompt_tokens}")
            print(f"Completion Tokens: {callback.completion_tokens}")
            print(f"Total Tokens: {callback.total_tokens}")

thanks for commenting @bashimr, which new example did you look at?

the latest is the one with function_filter, which is aligned with how the filters are being implemented in dotnet, that will make the above look like this:

class Filter:
  async def function_filter(context, next):
    await next(context)
    if context.result:
      print(f"Prompt tokens: {context.result.metadata.get('usage', {}).get('prompt_tokens', 'unknown')}")
      ...

I would love to hear your thoughts on that one!

FYI @matthewbolanos

[bashimr]

@eavanvalkenburg @moonbox3. I have a general feedback on the hooks implementation. From my point of view as an LLM developer, the most common use case for this is to be able to get the token usage after invoking a semantic function or executing an agent (with multiple tools invocations). I feel that the mechanism mentioned in the example is unnecessarily complicated, specially for the inexperienced python developers. I would recommend adopting the with-statement context as it is done in Langchain. So, getting the tokens after an LLM call is as easy as this:

        with get_openai_callback() as callback:
            response = agent.invoke({"input": "What is 2+2"})
            print(f"Prompt Tokens: {callback.prompt_tokens}")
            print(f"Completion Tokens: {callback.completion_tokens}")
            print(f"Total Tokens: {callback.total_tokens}")

thanks for commenting @bashimr, which new example did you look at?

the latest is the one with function_filter, which is aligned with how the filters are being implemented in dotnet, that will make the above look like this:

class Filter:
  async def function_filter(context, next):
    await next(context)
    if context.result:
      print(f"Prompt tokens: {context.result.metadata.get('usage', {}).get('prompt_tokens', 'unknown')}")
      ...

I would love to hear your thoughts on that one!

FYI @matthewbolanos

@eavanvalkenburg. Thanks for pointing this out. This is simpler to implement. But, to me, it does not feel very Pythonic. Looks more like dotnet paradigm which probably makes sense as the goal is to achieve parity with dotnet. Python developers are very familiar with 'with-statement' so Langchain's way of implementing callback to get token counts and doing other operations feels a bit more natural from Python's perspective.

[markwallace-microsoft]

Python 3.10 Test Coverage Report •

File	Stmts	Miss	Cover	Missing
semantic_kernel
kernel.py	262	50	81%	123, 125, 191, 198–207, 237, 247–255, 327–363, 433–434, 661, 682–684, 720, 722, 732
semantic_kernel/connectors/ai/open_ai/services
azure_config_base.py	34	4	88%	73–75, 90
open_ai_chat_completion_base.py	211	93	56%	97, 117, 142–146, 170, 174, 190–195, 211–240, 243–254, 272–279, 290–298, 314–321, 342, 350, 356–362, 374–380, 411, 450, 452–453, 458–463, 467–474, 496, 505–514
semantic_kernel/contents
function_result_content.py	57	11	81%	51, 56, 66, 81, 106, 111, 118–122
semantic_kernel/functions
kernel_function.py	83	2	98%	146, 166
kernel_function_from_method.py	81	4	95%	145–146, 152–153
kernel_function_from_prompt.py	157	7	96%	149–150, 171, 191, 216, 237, 317
semantic_kernel/planners/function_calling_stepwise_planner
function_calling_stepwise_planner.py	107	39	64%	126–211, 245–250
semantic_kernel/prompt_template/utils
template_function_helpers.py	31	1	97%	35
TOTAL	6172	1032	83%

Python 3.10 Unit Test Overview

Tests	Skipped	Failures	Errors	Time
1328	1 :zzz:	0 :x:	0 :fire:	12.639s :stopwatch: