Resource "azurerm_monitor_scheduled_query_rules_alert_v2" created but corrupted

[admsallai]

Is there an existing issue for this?

• [X] I have searched the existing issues

Community Note

• Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.8.1

AzureRM Provider Version

3.83.0

Affected Resource(s)/Data Source(s)

azurerm_monitor_scheduled_query_rules_alert_v2

Terraform Configuration Files

resource "azurerm_monitor_scheduled_query_rules_alert_v2" "total_ingestion_alert" {  
  name = "ingestion-alert"
  location = data.azurerm_resource_group.infrastructure_rg.location
  resource_group_name  = data.azurerm_resource_group.infra_rg.name
  scopes = [data.azurerm_log_analytics_workspace.law.id]
  severity = 2
  evaluation_frequency = "PT6H"
  window_duration = "P1D"
  description = "n/A"
  action {
    action_groups = [azurerm_monitor_action_group.action_group["actiongroup"].id]
  }
  criteria {
    operator = "GreaterThan"
    query = "Usage | where TimeGenerated > ago(24h) | summarize TotalIngestionVolGB = sum(Quantity)/1024.0"
    metric_measure_column = "TotalIngestionVolGB"
    time_aggregation_method = "Total"
    threshold = 5
  }
}

Debug Output/Panic Output

N/A

Expected Behaviour

Create an alert which is not corrupted.

Actual Behaviour

Alert is successfully created and visible in Azure Portal. However it is not being fired when criteria are met, and when trying to edit it in the portal this happend:

No summary is present. Unable to click Review+Save. Next step is Review+Save, even when Review+Save page is already open.

After contacting Microsoft support, they could not fix the issue, and this is how they described this issue after investigating:

"If an alert rule is created without managed identity from portal, we don't see that entry in the export template, whereas the terraform includes a Null value even though managed Identity is not enabled. Kindly ask you to reach out to terraform support and highlight this issue, they should probably fix it from their end."

This alert was working fine when creating 2 months ago, however when I need to recreate it again, with the same terraform code, this issue happened. This issue is present for all the alerts of this kind, not only this one.

Steps to Reproduce

1. terraform plan
2. terraform apply

Important Factoids

N/A

References

No response

[Chambras]

@admsallai interesting, have you tried adding that identity block and set it to SystemAssigned ? you might also upgrade the provider to the latest one 3.102.0 3.83.0 is kind of old...

[teowa]

Hi @admsallai ,

This alert was working fine when creating 2 months ago, however when I need to recreate it again, with the same terraform code, this issue happened. This issue is present for all the alerts of this kind, not only this one.

Since you are using same terraform code and same Terraform provider version but it failed, it might be some changes happened at service backend.

From the support, seems you need to add identity to solve this, the identity is introduced in version 3.98.0 from https://github.com/hashicorp/terraform-provider-azurerm/pull/25365, could you please try to use the latest version of the provider? Thanks.

[admsallai]

The updating of provider itself did not help, however then adding the identity block did! However, it is not possible to not have identity enabled, since adding this identity block enables it automatically and now permission needs to be assigned for reading logs in log analytics workspace.

[guidojw]

I'm also stuck on this issue with a log v2 alert that has an Application Insights instance as scope.

Do you know what role is needed for the alert identity to be able to read metrics/logs on Application Insights? Edit: found out Monitoring Reader does the trick 👍🏿

~~The alert history now shows the alert correctly triggers, but it also says the action group is triggered while I do not see this is actually happening. Anyone experiencing the same?~~ Edit: I found the culprit elsewhere

[JoshWeepie]

@teowa Terraform is passing identity[type:None, userAssignedIdentities:null] in the requestbody, but creating an alert manually in the portal without identity doesn't pass any identity parameters in the requestbody, according the Portal Activity log JSON.

The Azure log query docs say when you leave the radio as 'Default' the alert rule permissions are based on the permissions of the last user to edit the rule https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-create-log-alert-rule#configure-the-alert-rule-details

Essentially, anyone trying to create azurerm_monitor_scheduled_query_rules_alert_v2 without a managed identity can't until this is fixed, or they have to do it manually in the portal. If the portal/API supports creation without managed identity, shouldn't the azurerm provider behave the same for azurerm_monitor_scheduled_query_rules_alert_v2?

[JoshWeepie]

The API doc says identity is optional.

https://learn.microsoft.com/en-us/rest/api/monitor/scheduled-query-rules/create-or-update?view=rest-monitor-2023-03-15-preview&tabs=HTTP#request-body

The portal/API lets you create log query alerts without identity. The provider change https://github.com/hashicorp/terraform-provider-azurerm/pull/25365 is wrong for sending identity in the request parameters no matter what, is it not?

[JoshWeepie]

@admsallai @guidojw A workaround is to use the Azure CLI to update the --disabled or --severity properties on the log alert. You don't even need to set disabled true and then re-enable, you can just send a '--disabled false' and the alert will fix itself. Then you can edit the alert in the portal and save like normal as well. This will also fix the alert triggering and my alert worked again after modifying. 👍 💯

az monitor scheduled-query update --disabled true --name 'Scheduled log alert name' --resource-group rgname

or

az monitor scheduled-query update --severity 3 --name 'Scheduled log alert name' --resource-group rgname

[the-rule]

Having the same issue with hashicorp/azurerm v3.106.1

Would very much appreciate the fix for it since we manage a lot of monitoring alerts and manual involvement is not an option at all.

[admsallai]

I fixed the issue (or better to say created a workoround) by creating one user managed identity with necessary RBAC (Log Analytics Reader in my case) and assigned this identity to all alerts. Now it is all up and running without any issues.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.