Add support for large language model diagnostics in API and common modules

[suzuki-shm]

Resolves #785

---

This pull request adds support for configuring and testing Large Language Model (LLM) diagnostics in the API and workspace diagnostic systems. The changes introduce new model and contract types to represent LLM settings, update test data generation, and refactor integration test helpers to handle these new structures.

Details

Large Language Model diagnostics support

• Added new contract types (LargeLanguageModelSettings and LargeLanguageModelMessageSettings) to ApiDiagnostic.cs, Diagnostic.cs, and WorkspaceDiagnostic.cs for serializing LLM diagnostic options, including logs, requests, and responses.
• Extended the main diagnostic models (ApiDiagnosticModel, DiagnosticModel) and their test generators to include an optional LargeLanguageModel property and generation logic for LLM settings.

Integration test refactoring

• Refactored integration test helpers in ApiDiagnostic.cs and Diagnostic.cs to use a new MapToDto function, which maps the extended model to DTOs including LLM settings, replacing the previous getDto implementation.
• Updated normalization logic in integration tests to handle LLM settings, ensuring consistent test output and comparison for diagnostics with LLM configuration.

Test data generation

• Updated generators for diagnostic model updates and DTO overrides to include LLM settings, enabling richer test coverage for diagnostics involving LLMs.

Results

Here is the screenshot of extracted properties from my APIM resource.

[suzuki-shm]

@microsoft-github-policy-service agree

[waelkdouh]

@suzuki-shm can you please create a new PR to carry the changes against the new code base which is located in the V7 feature branch?

[guythetechie]

Thanks for the great PR. Unfortunately, we intentionally don't support this yet in ApiOps.

LLM diagnostics are only available in a preview version (2025-03-01-preview) of the APIM REST API. The latest GA version (2024-05-01) doesn't have this.

Once it's GA, we will quickly add support for this in ApiOps. We've had issues with preview APIs before, and as a rule of thumb, we avoid supporting them.

I'll update the issue with a workaround.