.Net: Bug: Not able to get AzureOpenAIChat to work on Android using MAUI

[cwbrandsdal]

Describe the bug I have been trying to get the semantic kernel azure openai to work in a .net MAUI project running on android without luck. The code runs just fine when running the windows app, but it always fails on Android both in emulator and on device.

To Reproduce Steps to reproduce the behavior: I have set up a boilerplate .net 8 MAUI project using xaml. The only thing I have done is added the semantic kernel NuGet and pasted the following code in the MainPage.xaml.cs:

public partial class MainPage : ContentPage
{
    public MainPage()
    {
        InitializeComponent();
    }

    private async Task<string> SendStringToAI()
    {
        string openAIKey = "REDACTED";
        string openAIEndpoint = "https://REDACTED.openai.azure.com/";
        string engine = "gpt-4o";

        try
        {
            AzureOpenAIChatCompletionService chatCompletionService = new (engine,openAIEndpoint, openAIKey);

            var res = await chatCompletionService.GetChatMessageContentsAsync("This is just a test.");

            return res.ToString();

        }
        catch (Exception ex)
        {
            string err = ex.Message;
        }

        return "No response";
    }

    private async void OnSendClicked(object sender, EventArgs e)
    {
        var response = await SendStringToAI();
        ResponseLabel.Text = response;
    }
}

This works fine on Windows, but on android I always get the following error on the call to GetChatMessageContentsAsync:

{Microsoft.SemanticKernel.HttpOperationException: Invalid value for 'content': expected a string, got null.
Status: 400 (model_error)

Content:
{
  "error": {
    "message": "Invalid value for 'content': expected a string, got null.",
    "type": "invalid_request_error",
    "param": "messages.[0].content",
    "code": null
  }
}

Headers:
x-ms-region: REDACTED
apim-request-id: REDACTED
x-ratelimit-remaining-requests: REDACTED
x-ms-rai-invoked: REDACTED
X-Request-ID: REDACTED
ms-azureml-model-error-reason: REDACTED
ms-azureml-model-error-statuscode: REDACTED
Strict-Transport-Security: REDACTED
azureml-model-session: REDACTED
X-Content-Type-Options: REDACTED
x-envoy-upstream-service-time: REDACTED
x-ms-client-request-id: 4359a357-f2f7-469f-911d-7e7cc3d95efc
x-ratelimit-remaining-tokens: REDACTED
Date: Thu, 27 Jun 2024 19:44:02 GMT
Content-Length: 187
Content-Type: application/json

 ---> Azure.RequestFailedException: Invalid value for 'content': expected a string, got null.
Status: 400 (model_error)

   at Azure.Core.HttpPipelineExtensions.ProcessMessageAsync(HttpPipeline pipeline, HttpMessage message, RequestContext requestContext, CancellationToken cancellationToken)
   at Azure.AI.OpenAI.OpenAIClient.GetChatCompletionsAsync(ChatCompletionsOptions chatCompletionsOptions, CancellationToken cancellationToken)
   at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.<RunRequestAsync>d__55`1[[Azure.Response`1[[Azure.AI.OpenAI.ChatCompletions, Azure.AI.OpenAI, Version=1.0.0.0, Culture=neutral, PublicKeyToken=92742159e12e44c8]], Azure.Core, Version=1.39.0.0, Culture=neutral, PublicKeyToken=92742159e12e44c8]].MoveNext()
   --- End of inner exception stack trace ---
   at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.<RunRequestAsync>d__55`1[[Azure.Response`1[[Azure.AI.OpenAI.ChatCompletions, Azure.AI.OpenAI, Version=1.0.0.0, Culture=neutral, PublicKeyToken=92742159e12e44c8]], Azure.Core, Version=1.39.0.0, Culture=neutral, PublicKeyToken=92742159e12e44c8]].MoveNext()
   at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.GetChatMessageContentsAsync(ChatHistory chat, PromptExecutionSettings executionSettings, Kernel kernel, CancellationToken cancellationToken)
   at Test.MainPage.SendStringToAI() in D:\Temp\Test\MainPage.xaml.cs:line 29}

{
  "error": {
    "message": "Invalid value for 'content': expected a string, got null.",
    "type": "invalid_request_error",
    "param": "messages.[0].content",
    "code": null
  }
}

I have tried to set it up with DI and Kernel, but exactly the same result. Works on Windows, not on Android.

Expected behavior I expect it to just give a simple return string.

Platform

• OS: Windows
• IDE: Visual Studio
• Language: C#
• Source: NuGet package Microsoft.SemanticKernel 1.15.0

[siddharthsingh89]

facing the same issue. It seems like it is not able to send the payload in the request to openai on android and iOS. Any workarounds?

[github-actions[bot]]

This issue is stale because it has been open for 90 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.

[sphenry]

Additional repro here: https://github.com/arafattehsin/parkinggpt

[markwallace-microsoft]

And here: https://github.com/microsoft/semantic-kernel/discussions/10029

[rogerbarreto]

After investigating further I wasn't able to reproduce the error and everything worked as expected below:

The provided Code wasn't complete, here's what I did.

1. Installed the latest version of MAUI for Visual Studio 2022
2. Created a new MAUI empty project for .NET 9.
3. Installed Microsoft.SemanticKernel package.
4. Update MainPage.xaml.cs (code below)
5. Update MainPage.xaml (code below)
6. Set Android as starting application
7. Run the Application (New fresh android installation was necessary from the emulator 1

MainPage.xaml

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://schemas.microsoft.com/dotnet/2021/maui"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="MauiApp1.MainPage">

    <ScrollView>
        <VerticalStackLayout
            Padding="30,0"
            Spacing="25">
            <Image
                Source="dotnet_bot.png"
                HeightRequest="185"
                Aspect="AspectFit"
                SemanticProperties.Description="dot net bot in a hovercraft number nine" />

            <Label
                Text="Hello, World!"
                Style="{StaticResource Headline}"
                SemanticProperties.HeadingLevel="Level1" />

            <Label
                Text="Welcome to &#10;.NET Multi-platform App UI"
                Style="{StaticResource SubHeadline}"
                SemanticProperties.HeadingLevel="Level2"
                SemanticProperties.Description="Welcome to dot net Multi platform App U I" />

            <Button
                x:Name="ResponseBtn"
                Text="Click me" 
                SemanticProperties.Hint="Gets the response from the LLM"
                Clicked="OnResponseClicked"
                HorizontalOptions="Fill" />

            <Label
                x:Name="ResponseLabel"
                Text="Click for LLM response"
                Style="{StaticResource Body}"
                HorizontalOptions="Center"
                SemanticProperties.Description="Counter label" />
        </VerticalStackLayout>
    </ScrollView>

</ContentPage>

[!NOTE] Changed slightly the script

• Added namespace top page missing portion
• Usage of the GetChatMessageContentAsync extension method instead.
• Rename endpoint to modelId
• Returned the error message to be displayed in the UI (if any)
• OnSend -> OnResponse

MainPage.xaml.cs

using Microsoft.SemanticKernel.Connectors.AzureOpenAI;
using Microsoft.SemanticKernel.ChatCompletion;

namespace MauiApp1
{
    public partial class MainPage : ContentPage
    {
        public MainPage()
        {
            InitializeComponent();
        }

        private async Task<string> SendStringToAI()
        {
            string openAIEndpoint = "https://<redacted>.azure.com/";
            string deploymentName = "gpt-4o";
            string apiKey = "redacted";

            try
            {
                AzureOpenAIChatCompletionService chatCompletionService = new(deploymentName, openAIEndpoint, apiKey);

                var res = await chatCompletionService.GetChatMessageContentAsync("This is just a test.")!;

                return res.ToString();

            }
            catch (Exception ex)
            {
                return ex.Message;
            }
        }

        private async void OnResponseClicked(object sender, EventArgs e)
        {
            var response = await SendStringToAI();
            this.ResponseLabel.Text = response;
        }
    }
}

[arafattehsin]

I did a bit of digging and I can tell you that the above sample works well. However, the above code doesn't talk about:

• JSON Schema
• Image Content

Therefore, the problem still exists in my app. I will try to repro it using the similar sample and will let you all know here.

[arafattehsin]

I've posted an update here: https://github.com/microsoft/semantic-kernel/issues/10082#issuecomment-2603320465

[sophialagerkranspandey]

Tagging @RogerBarreto for review with the new information from @arafattehsin. Roger I saw you weren't able to repo the issue but Arafat has added further details.

[rogerbarreto]

Thanks @arafattehsin for the updated, will dig a bit further into that.

[sjpulta]

I have this code in my .NET MAUI but it always result to "The SSL connection could not be established." This is my reference in building my Android App with GenerativeAI using Semantic Kernel.

Can somebody help to fix this? I am using OpenAI as my GenAI.

public partial class HomePage : ContentPage { private readonly Kernel _kernel; private readonly HttpClient _httpClient;

public HomePage()//Kernel kernel
{
    InitializeComponent();

    var builder = MauiApp.CreateBuilder();
    builder.Services.AddKernel().AddOpenAIChatCompletion("gpt-4", myAPIKey);
    var app = builder.Build();

    _kernel = app.Services.GetRequiredService<Kernel>();
}

private async void aisearch_clicked(object sender, EventArgs e)
{
    string question = aisearch.Text;
    responselabel.Text = "Processing...";

    try
    {
        await foreach (var response in _kernel.InvokePromptStreamingAsync<string>(question))
        {
            responselabel.Text += response;
        }
    }
    catch (Exception ex)
    {
        responselabel.Text = $"Error: {ex.Message}";
    }
}

}

[Asttear]

I have this code in my .NET MAUI but it always result to "The SSL connection could not be established."

Manually adding the service with a new HttpClient resolves the issue.

services.AddSingleton<IChatCompletionService>(_ =>
    new OpenAIChatCompletionService(
        modelId: "gpt-4",
        apiKey: "<key>",
        httpClient: new HttpClient()
    )
);

[rogerbarreto]

I did a bit of digging and I can tell you that the above sample works well. However, the above code doesn't talk about:

• JSON Schema
• Image Content

Therefore, the problem still exists in my app. I will try to repro it using the similar sample and will let you all know here.

@arafattehsin Are you still having issues on your demo? If so please share your example using the Schema and Images, will be easier to spot any problems.

[arafattehsin]

@RogerBarreto No, I can see that it is working fine and I do not have any issues. Thanks for checking in.