documentdb.DatabaseAccount : Can not update the list of capabilities

[MatthewBettonMaif]

What happened?

After creating a new documentdb.DatabaseAccount with some capabilities (like DisableRateLimitingResponses, EnableMongo and EnableUniqueCompoundNestedDocs) we can not update those capabilities with a new one (like EnableMongoRetryableWrites) with pulumi azure-native.

The code has changed from :

    capabilities: [
        {
            name: 'DisableRateLimitingResponses',
        },
        {
            name: 'EnableMongo',
        },
        {
            name: 'EnableUniqueCompoundNestedDocs',
        },
    ],

to this

    capabilities: [
        {
            name: 'DisableRateLimitingResponses',
        },
        {
            name: 'EnableMongo',
        },
        {
            name: 'EnableUniqueCompoundNestedDocs',
        },
        {
            name: 'EnableMongoRetryableWrites',
        },
    ],

The pulumi preview shows us that an update is needed.

The pulumi up outputs one updated resource.

But then the az cli command az cosmosdb show --resource-group rg-cosmosdb-test-sandbox-compliant --name compliant-mongodb | jq ".capabilities" and ARM template got from Azure portal do not indicate a change.

If we made a change on the capabilities from az cli, a pulumi refresh does not get the change.

Expected Behavior

Pulumi should updates the list of capabilities as it indicates us it has been during preview and update.

It works with Azure API Rest

PATCH on https://management.azure.com/subscriptions/xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx/resourceGroups/rg-test-databases-sandbox/providers/Microsoft.DocumentDB/databaseAccounts/test-mongodb-v42-build3?api-version=2022-02-15-preview

or

PATCH on https://management.azure.com/subscriptions/xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx/resourceGroups/rg-test-databases-sandbox/providers/Microsoft.DocumentDB/databaseAccounts/test-mongodb-v42-build3?api-version=2021-03-15

( I have tested 2 API versions )

With body

{
    "properties": {
        "capabilities": [
            {
                "name": "EnableMongo"
            },
            {
                "name": "EnableMongoRetryableWrites"
            },
            {
                "name": "DisableRateLimitingResponses"
            },
            {
                "name": "EnableUniqueCompoundNestedDocs"
            }
        ]
    }
}

Steps to reproduce

Pulumi up with the code below and just 3 of the four capabilities, then another pulumi up with the four capabilities.

index.ts

import * as pulumi from '@pulumi/pulumi'
import * as documentdb from '@pulumi/azure-native/documentdb'
import * as resources from '@pulumi/azure-native/resources'

export const env = pulumi.getStack()
export const projectName = pulumi.getProject()
export const rootResourceGroupName = `rg-${projectName}-${env}`

export const compliantResourceGroupName = `${rootResourceGroupName}-compliant`

export const resourceGroupCompliant = new resources.ResourceGroup(compliantResourceGroupName, {
    resourceGroupName: compliantResourceGroupName,
})

const databaseAccountName = `compliant-mongodb`

new documentdb.DatabaseAccount(databaseAccountName, {
    accountName: databaseAccountName,
    resourceGroupName: resourceGroupCompliant.name,
    kind: documentdb.DatabaseAccountKind.MongoDB,
    tags: {},
    apiProperties: {
        serverVersion: documentdb.ServerVersion.ServerVersion_4_0,
    },
    backupPolicy: {
        type: documentdb.BackupPolicyType.Periodic,
        periodicModeProperties: {
            backupIntervalInMinutes: 240,
            backupRetentionIntervalInHours: 8,
        },
    },
    consistencyPolicy: {
        defaultConsistencyLevel: documentdb.DefaultConsistencyLevel.BoundedStaleness,
        maxIntervalInSeconds: 10,
        maxStalenessPrefix: 200,
    },
    capabilities: [
        {
            name: 'DisableRateLimitingResponses',
        },
        {
            name: 'EnableMongo',
        },
        {
            name: 'EnableUniqueCompoundNestedDocs',
        },
        {
            name: 'EnableMongoRetryableWrites',
        },
    ],
    cors: [
        {
            allowedOrigins: '*',
        },
    ],
    databaseAccountOfferType: documentdb.DatabaseAccountOfferType.Standard,
    enableAnalyticalStorage: false,
    enableFreeTier: false,
    locations: [
        {
            failoverPriority: 0,
            isZoneRedundant: true,
            locationName: 'francecentral',
        },
    ],
    networkAclBypass: documentdb.NetworkAclBypass.None,
    publicNetworkAccess: documentdb.PublicNetworkAccess.Disabled,
    isVirtualNetworkFilterEnabled: true,
})

Output of pulumi about

CLI
Version      3.55.0
Go Version   go1.19.5
Go Compiler  gc

Plugins
NAME    VERSION
nodejs  unknown

Host
OS       darwin
Version  13.2.1
Arch     x86_64

This project is written in nodejs: executable='/usr/local/bin/node' version='v19.1.0'

Current Stack: MAIF/policies-cosmosdb-test/sandbox

TYPE                                                      URN
pulumi:pulumi:Stack                                       urn:pulumi:sandbox::policies-cosmosdb-test::pulumi:pulumi:Stack::policies-cosmosdb-test-sandbox
pulumi:providers:azure-native                             urn:pulumi:sandbox::policies-cosmosdb-test::pulumi:providers:azure-native::default_1_88_1
azure-native:resources:ResourceGroup                      urn:pulumi:sandbox::policies-cosmosdb-test::azure-native:resources:ResourceGroup::rg-policies-cosmosdb-test-sandbox-compliant
azure-native:resources:ResourceGroup                      urn:pulumi:sandbox::policies-cosmosdb-test::azure-native:resources:ResourceGroup::rg-policies-cosmosdb-test-sandbox-non-compliant
azure-native:network:VirtualNetwork                       urn:pulumi:sandbox::policies-cosmosdb-test::azure-native:network:VirtualNetwork::vnet-policies-cosmosdb-test-sandbox
azure-native:network:Subnet                               urn:pulumi:sandbox::policies-cosmosdb-test::azure-native:network:Subnet::snet-policies-cosmosdb-test-sandbox
azure-native:network:Subnet                               urn:pulumi:sandbox::policies-cosmosdb-test::azure-native:network:Subnet::snet-private-policies-cosmosdb-test-sandbox
azure-native:documentdb/v20220815preview:DatabaseAccount  urn:pulumi:sandbox::policies-cosmosdb-test::azure-native:documentdb/v20220815preview:DatabaseAccount::compliant-mongodb
azure-native:network:PrivateEndpoint                      urn:pulumi:sandbox::policies-cosmosdb-test::azure-native:network:PrivateEndpoint::pe-compliant-mongodb


Found no pending operations associated with sandbox

Backend
Name           pulumi.com
URL            https://app.pulumi.com/MAIF
User           MAIF
Organizations  MAIF, freetrialteamproedition

Additional context

Pulumi.sandbox.yaml

config:
    azure:environment: public
    azure:location: FranceCentral
    azure-native:subscriptionId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxxx
    azure-native:tenantId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxxx
    azure-native:location: FranceCentral

Pulumi.yaml

name: cosmosdb-test
runtime: nodejs
description: A minimal Azure Native TypeScript Pulumi program

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[thomas11]

Hi @MatthewBettonMaif, thank you for the very detail bug report! I could easily reproduce it. I'll sync with the team on this issue.

[thomas11]

In the provider's schema (schema.json line 393,495), the definition of capabilities is

                "capabilities": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "$ref": "#/types/azure-native:documentdb:CapabilityResponse"
                    },
                    "description": "List of Cosmos DB capabilities for the account"
                },

Suspicious that the type is a ...Response type when we actually have an azure-native:documentdb:Capability type defined.

[mikhailshilkov]

I check TypeScript types and the capabilities seem to be defined correctly

    capabilities?: pulumi.Input<pulumi.Input<inputs.documentdb.CapabilityArgs>[]>;

I logged the request body and it contained all four capabilities on the second update:

{
  "kind": "MongoDB",
  "location": "westeurope",
  "properties": {
    "apiProperties": {
      "serverVersion": "4.0"
    },
    "backupPolicy": {
      "periodicModeProperties": {
        "backupIntervalInMinutes": 240,
        "backupRetentionIntervalInHours": 8
      },
      "type": "Periodic"
    },
    "capabilities": [
      {
        "name": "DisableRateLimitingResponses"
      },
      {
        "name": "EnableMongo"
      },
      {
        "name": "EnableUniqueCompoundNestedDocs"
      },
      {
        "name": "EnableMongoRetryableWrites"
      }
    ],
    "consistencyPolicy": {
      "defaultConsistencyLevel": "BoundedStaleness",
      "maxIntervalInSeconds": 10,
      "maxStalenessPrefix": 200
    },
    "cors": [
      {
        "allowedOrigins": "*"
      }
    ],
    "createMode": "Default",
    "databaseAccountOfferType": "Standard",
    "enableAnalyticalStorage": false,
    "enableFreeTier": false,
    "isVirtualNetworkFilterEnabled": true,
    "locations": [
      {
        "failoverPriority": 0,
        "isZoneRedundant": true,
        "locationName": "francecentral"
      }
    ],
    "networkAclBypass": "None",
    "publicNetworkAccess": "Disabled"
  },
  "tags": {
    
  }
}

Yet, the CLI command reports three capabilities still. This looks like an upstream bug to me.

[thomas11]

I just tried a few variations of this repro with the latest Azure Native 2.21.2 but always got the same result. Capabilities cannot be added nor removed with PUT. I guess that's one of the things they're referring to with the vague The "Update" method is preferred when performing updates on an account statement.