semantic-kernel Python: Bug: retry

Describe the bug The Kernel class has a property retry_mechnism.

This is intended to be set to a Implementation of RetryMechanismBase from the semantic_kernel.reliability extension to handle the retry logic. However this property seems to never be used, leaving us unable to handle retries in the intended way. This was tested both with the kernel.invoke functions as well as with get_chat_message_content.

To Reproduce

Create Implementation of RetryMechanismBase
Initialize in Kernel
Run Chat Completion or function invoke Code Sample to reproduce:

from semantic_kernel import Kernel
from semantic_kernel.reliability.retry_mechanism_base import RetryMechanismBase
from typing import Awaitable, Callable, TypeVar
from semantic_kernel.connectors.ai.open_ai import (
    AzureChatCompletion,
)
from semantic_kernel.contents.chat_history import ChatHistory
from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings
import pytest

T = TypeVar("T")

azure_openai_endpoint = "your-endpoint"
azure_openai_key = "your-key"
azure_openai_model = "gpt-4o"

class FailingRetryMechanism(RetryMechanismBase):
    async def execute_with_retry(self, action: Callable[[], Awaitable[T]]) -> T:
        raise NotImplementedError

kernel = Kernel(retry_mechanism=FailingRetryMechanism())
kernel.add_service(
            AzureChatCompletion(
                deployment_name=azure_openai_model,
                endpoint=azure_openai_endpoint,
                service_id=azure_openai_model,
                api_key=azure_openai_key,
            ),
        )

chat_completion_service = kernel.get_service(service_id=azure_openai_model)

chat_history = ChatHistory()
chat_history.add_user_message("Hello, how are you?")

with pytest.raises(NotImplementedError):
    response = await chat_completion_service.get_chat_message_content(
        chat_history=chat_history,
        settings = PromptExecutionSettings(),
    )

The test above fails, because FailingRetryMechanism is never used.

Expected behavior The provided retry mechanism should be used..

Platform

OS: Windows, Mac
IDE: VS Code
Language: Python
Source:package version 1.9.0

Sep 27 '24 13:09 CBumann

@TaoChenOSU could you have a look at this, please?

Sep 30 '24 13:09 moonbox3

Is there anything I can clarify or adjust to support you better here?

Oct 25 '24 08:10 CBumann

Hi @CBumann, the retry_mechanism was a feature that was never implemented when we were trying to reach parity with Semantic Kernel .Net. Currently the .Net SDK no longer supports this out-of-the-box retry policy, we decided to deprecate it in Semantic Kernel Python.

The recommended way to perform retry on a kernel function is now through filters. You can refer to this new sample here: https://github.com/microsoft/semantic-kernel/blob/main/python/samples/concepts/filtering/retry_with_filters.py

Please let us know if using a filter is enough for your scenario and what additional features you'd like to see in SK!

Dec 16 '24 17:12 TaoChenOSU

Python: Bug: retry_machanism is not used