Feature Request - Add support for Resource-specific mode for diagnostic settings/resource logs

[janegilring]

Additional Related Issues

• #1283

Context

Azure resource logs are platform logs that provide insight into operations that were performed within an Azure resource. The content of resource logs varies by the Azure service and resource type. Resource logs aren't collected by default. This issue discusses the diagnostic setting required for each Azure resource to send its resource logs to Log Analytics workspaces.

The documentation also states the following:

All Azure services eventually will use the resource-specific mode. As part of this transition, some resources allow you to select a mode in the diagnostic setting. Specify resource-specific mode for any new diagnostic settings because this mode makes the data easier to manage. It also might help you avoid complex migrations later.

Customers wants to start leveraging the new mode so that data is written to individual tables for each category of the resource - as this is making it easier to work with the logs in queries, alerts and workbooks - as well as overcoming the 500 column limit.

Request

Either implement new custom policies in the CAF reference architecture or add support in the existing "Deploy Diagnostic Settings for..." policies by offering input variables to make it possible to opt-in for Resource-specific mode.

When policy support is in place, variables in the various accelerators (Bicep, Terraform++) should also make it possible to opt-in for Resource-specific mode.

Even though not all Azure services supports the new mode as of now, it could make sense to start implementing support for it for the services that do support it.

[jtracey93]

Thanks @janegilring for the issue.

We are working closely with the Diagnostic Settings team on this and some policies as we speak. We also updated our FAQ with this here: https://github.com/Azure/Enterprise-Scale/wiki/FAQ#the-azurediagnostics-table-in-my-log-analytics-workspace-has-hit-the-500-column-limit-what-should-i-do

Stay tuned 👍

[paulgrimley]

Hi @janegilring we have raised this with the respective product team owners and this is a complex operation as each RP needs to support this mode and this will take time, then once the respective RP supports the resource specific mode the Azure Policy will require supporting this (as Jack mentioned there is a parallel effort to provide built-in policies so this could form part of that work for services that have this capability). I am also asking the Azure Monitor product team if they can publish the RP's that support resource specific mode - TBC