Foundry returns 500 status code when using ResponseFormat (aka structured response) in ChatClient options

[tomlm]

AB#66980

[jonwis]

Can you include what model you were using? The output of (await manager.ListLoadedModelsAsync()).Select(_ => _.ModelId); for instance?

[tomlm]

Here's repo code:

   public static async Task Main(string[] args)
   {
       var app = new TestApp(); // Create an instance of TestApp  

       var modelName = "Phi-4-mini-instruct-cuda-gpu";
       var manager = new FoundryManager();
       await manager.StartServiceAsync();
       var modelInfo = await manager.GetModelInfoAsync(modelName);
       Console.WriteLine($"Model: {modelInfo.Alias} ({modelInfo.ModelId}) {modelInfo.Version}");
       await manager.LoadModelAsync(modelName);

       var schema =
           $$"""
           {
             "type": "object",
             "additionalProperties": false,
             "required": [
               "Explanation",
               "Result"
             ],
             "properties": {
               "Explanation": {
                 "type": [
                   "string",
                   "null"
                 ],
                 "description": "Explain your reasoning"
               },
               "Result": {
                 "type": [
                   "array",
                   "null"
                 ],
                 "items": {
                   "type": "string"
                 },
                 "description": "The result"
               }
             }
           }
           """;
       var responseFormat = ChatResponseFormat.CreateJsonSchemaFormat("result", 
           jsonSchema: BinaryData.FromString(schema), 
           jsonSchemaFormatDescription: "An JSON object with two properties: Explanation and Result",
           jsonSchemaIsStrict: true);

       ApiKeyCredential key = new ApiKeyCredential(manager.ApiKey);
       OpenAIClient client = new OpenAIClient(key, new OpenAIClientOptions
       {
           Endpoint = manager.Endpoint
       });

       var chatClient = client.GetChatClient(modelInfo.ModelId);

       var result = chatClient.CompleteChat(["List five common male first names?"], new ChatCompletionOptions()
       {
           ResponseFormat = responseFormat 
       });
       Console.WriteLine(result.Value.Content[0].Text);
       Console.WriteLine("Press any key to exit...");
       Console.ReadKey(true);
   }

1. If you comment out the ResponseFormat = responseFormat line the local model returns text response correctly
2. If you change it to call OpenAI with following changes:

• set modelName to "gpt-4o"
• configure a real "OpenAI" key
• comment out Endpoint = manager.Endpoint The OpenAI response correctly returns a JSON object matching the schema.

This could be because either: a. the phi-4 model doesn't support structured output OR b. the schema is blowing up foundry.

[tomlm]

Also this seems to happen with

• phi-4-mini
• qwen2.5-0.5b
• qwen2.5-1.5b I've not tried more models than that.

[samuel100]

Duplicate of #112