Creating API Management product policy gives ValidationError

[cloud-devlpr]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Issue

XML policy applied successfully for azurerm_api_management_api_policy but fails when trying to apply at the product level azurerm_api_management_product_policy

Terraform (and AzureRM Provider) Version

 Terraform v0.12.18
 provider.azurerm v1.40.0

Affected Resource(s)

• azurerm_api_management_product_policy

Terraform Configuration Files

resource "azurerm_api_management_api_policy" "AllOperations" {
  api_name            = azurerm_api_management_api.testApi.name
  api_management_name = azurerm_api_management.apim.name
  resource_group_name = var.resourceGroupName
  xml_content         = file("../API Policies/AllOperations.xml")
}
resource "azurerm_api_management_product_policy" "testProduct" {
  product_id          = azurerm_api_management_product.testProduct.product_id
  api_management_name = azurerm_api_management.apim.name
  resource_group_name = var.resourceGroupName
  xml_content         = file("../API Policies/AllOperations.xml")
}

AllOperations.xml

<policies>
    <inbound>
        <base />
        <set-variable name="abc" value="@(context.Request.Headers.GetValueOrDefault("X-Header-Name", ""))" />
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

Expected Behavior

Policy is created for the product

Actual Behavior

Error creating or updating Product Policy: apimanagement.ProductPolicyClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="ValidationError" Message="One or more fields contain incorrect values:" Details=[{"code":"ValidationError","message":"'X-Header-Name' is an unexpected token. Expecting white space. Line 16, position 86.","target":"representation"}]

Steps to Reproduce

1. terraform apply

References

• A similar issue was raised to fix for the API policy #3918 and fixed by #4140 . Assuming same fix needs to be applied at the API Product level?

[cloud-devlpr]

@tombuildsstuff should this be an easy fix based on the previous policy validation fix ?

[cloud-devlpr]

Any update on this please. I can confirm that the policy works from the Azure portal.

[evertonmc]

this works if you escape the sub-quotes to use &quot; <set-variable name="abc" value="@(context.Request.Headers.GetValueOrDefault(&quot;X-Header-Name&quot;, &quot;&quot;))" />

[adhoc-am]

I would try to change apim policy xml variable during terraform apply. for one string variable I had "ValidationError is an unexpected token" looks like terraform during apply add quotas to variable. For me help jsonencode function. like this someVar = jsonencode(var.replaceSomeVar), after that terraform applies xml changes well

[subesokun]

I also ran into the same problem and the recommendation from https://github.com/terraform-providers/terraform-provider-azurerm/issues/3918#issuecomment-544544215 helped me to fix my problem but I'm not very happy with the solution as I've to replace those characters manually and my policy file is now very hard to read. E.g.

<policies>
    <inbound>
        <base />
        <set-body>
        @{ 
            string inBody = context.Request.Body.As&#60;string&#62;(preserveContent: true); 
            return inBody; 
        }
        </set-body>
    </inbound>
    ...
</policies>

@tombuildsstuff Is there no way that the azurerm provider can perform this encoding on-the-fly on the content of every @(expression) or @{expression} statement? I could add a magic script in my pipeline to do the encoding for me but I'd like to avoid that as it just adds complexity. And I'm pretty sure everybody with a bit more complex policy will run into this issue and gives half-way up because this error is very difficult to understand/debug.

I'd expect that if I'm able to upload a specific policy via the Azure portal then this should be also possible for the azurerm provider without any additional modifications to the policy file.

Helpful link: https://docs.microsoft.com/en-us/azure/api-management/api-management-policy-expressions

[subesokun]

I wrote now an own encoder but this was quite tricky because this APIM policy you get out of the Azure portal is not a well-formed XML document. I guess in some corner cases my encoder will crash because it's pretty difficult to write a regex for this C# statements.

[subesokun]

Sorry, I didn't see the linked MR above. Only the APIM API policy resource is sending the XML policies in raw format, see https://github.com/terraform-providers/terraform-provider-azurerm/pull/4140/files#diff-dc0dac7a597c425e71a358da620e9112R101

This change needs to be applied on the APIM API operation policy resource too.

https://github.com/riordanp/terraform-provider-azurerm/blob/master/azurerm/resource_arm_api_management_product_policy.go#L77

https://github.com/riordanp/terraform-provider-azurerm/blob/master/azurerm/resource_arm_api_management_api_operation_policy.go#L79

[awnugent]

this works if you escape the sub-quotes to use &quot; <set-variable name="abc" value="@(context.Request.Headers.GetValueOrDefault(&quot;X-Header-Name&quot;, &quot;&quot;))" />

Note that line feeds (escape with 
) and conditionals such as "&&" (escape with &&) have the same issue. Probably others. HTH

[alexgagnon]

Agree with @subesokun. Would be very helpful to be able to send it in the same format we can enter it manually in the portal.

[smithimage]

Any progress with this? 🤸‍♂️

As reported above it is annoying enough that you have to escape all quotes etc. manually. What is even more disturbing is that once you have successfully managed to save the policy the portal un-escapes the policy rendering it impossible to update it without re-escaping the whole thing again.

Another thing that I have noticed is that the behavior differs between the policy elements. seems to handle escaping by itself while the does not. The following characters have I so far have had to escape manually.

" = " < = <

= >

Please fix or at least make the engine re-render the escaped characters as escaped once one has done the manual labour of escaping it.

As it is right now we have to keep a separate GIT repository with our policies, where we store them escaped and manually resave the whole thing when we need to update them.

[smithimage]

Another thing; the same behaviour is unfortunately found in the new VS Code Plugin 😢

https://github.com/microsoft/vscode-apimanagement

[LegacyChris]

I have gone through the other issue that has been closed and nothing seems to work for me. I have a relatively large policy file and this is really a pain. Any recommendations on this?