Unable to delete `defenderForStorage` resources

[stooj]

What happened?

When trying to delete a azure-native.security.DefenderForStorage resource with azure-native versions >= 2.49.0, the azure api returns an error:

error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="Properties can't be null"

In Azure Native versions < 2.49.0, the REST call is:

PUT https://management.azure.com/%2Fsubscriptions%2F***************%2FresourceGroups%2FDefenderStorageTeste8525af1%2Fproviders%2FMicrosoft.Storage%2FstorageAccounts%2Fdefendertest048a72fd/providers/Microsoft.Security/defenderForStorageSettings/current?api-version=2022-12-01-preview

Body:
{"properties":{"isEnabled":false,"overrideSubscriptionLevelSettings":true}}

But versions >= 2.49.0 are sent as the following:

PUT https://management.azure.com/subscriptions/*************/resourceGroups/DefenderStorageTestd0e1996e/providers/Microsoft.Storage/storageAccounts/defendertest5f49c189/providers/Microsoft.Security/defenderForStorageSettings/current?api-version=2022-12-01-preview

Body:
{}

Example

import getpass

import pulumi
from pulumi_azure_native import eventgrid, resources, security, storage

username = getpass.getuser()
case_id = "3472"

config = pulumi.Config()
location = config.get("location", "uksouth")

resource_group = resources.ResourceGroup(f"rg{username.title()}{case_id}")

account = storage.StorageAccount(
    f"sa{username}{case_id}",
    resource_group_name=resource_group.name,
    sku=storage.SkuArgs(
        name=storage.SkuName.STANDARD_LRS,
    ),
    kind=storage.Kind.STORAGE_V2,
)

topic = eventgrid.Topic(
    f"topic{username.title()}{case_id}",
    public_network_access=eventgrid.PublicNetworkAccess.ENABLED,
    resource_group_name=resource_group.name,
)

subscription_id = resource_group.id.apply(lambda id: id.split("/")[2])

defender_for_storage = security.DefenderForStorage(
    f"defenderForStorage{username.title()}{case_id}",
    properties={
        "is_enabled": True,
        "malware_scanning": {
            "on_upload": {
                "cap_gb_per_month": -1,
                "is_enabled": True,
            },
            "scan_results_event_grid_topic_resource_id": topic.id,
        },
        "override_subscription_level_settings": True,
        "sensitive_data_discovery": {
            "is_enabled": True,
        },
    },
    resource_id=account.id,
    setting_name="current",
)

1. pulumi up
2. pulumi destroy

To clean up after (because you can't destroy with pulumi)

1. Click-ops delete the storage account
2. pulumi refresh
3. pulumi delete

Output of pulumi about

CLI
Version      3.127.0
Go Version   go1.22.5
Go Compiler  gc

Plugins
KIND      NAME          VERSION
resource  azure-native  2.51.0
language  python        unknown

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[thomas11]

Very curious. Thank you for narrowing down the affected versions!

v2.49.0, where the problem starts, is when Enable TypedDict input types for the Python SDK by @julienp in https://github.com/pulumi/pulumi-azure-native/pull/3400 was merged. We'll have to look into that one.

[thomas11]

@stooj, did you change any of your code between using v2.49 of the provider and later versions?

[julienp]

In the release notes of 2.49.0 https://github.com/pulumi/pulumi-azure-native/releases/tag/v2.49.0 we also have

[danielrbradley]

I think this is likely related to default resource state - where the resource doesn't have a delete operation, so we try to 'put' a default value when doing a delete of the resource.

It looks like the default is defined here:

https://github.com/pulumi/pulumi-azure-native/blob/3a4638df1e9b832261333d784e19983dbfb76a6a/provider/pkg/openapi/defaults/defaultResourcesState.go#L50-L54

We need to diagnose why this isn't being sent, and an empty value is being sent instead.

[danielrbradley]

I think Julien's on to something - in the changes for v2.49.0 we had top level properties removed.

These came from this change in the upstream specifications where the x-ms-client-flatten was removed, therefore pushing the properties to be sub-properties:

• https://github.com/Azure/azure-rest-api-specs/pull/29276

Therefore, I think our default value no longer matches the shape of the schema, and so the properties are silently removed and not sent to the API.

[stooj]

did you change any of your code between using v2.49 of the provider and later versions?

@thomas11 I must have done 🤔 Let me see if I can recreate it.