Cannot import resources when provider has default tags

[piotr-bzdyl-vertexinc]

Describe what happened

When an AWS provider instance has defaultTags configured, AWS resources cannot be imported into Pulumi state using pulumi up because of differences shown by pulumi in the imported resource tags.

Sample program

$ cat requirements.txt
pulumi>=3.0.0,<4.0.0
pulumi-aws>=6.0.2,<7.0.0

$ cat Pulumi.yaml
name: pulumi-defaulttags-import-issue
runtime:
  name: python
  options:
    virtualenv: venv

$ cat __main__.py
import pulumi
import pulumi_aws

test_import = pulumi.Config().get_bool("test_import")

bucket_name = "pulumi-aws-defaulttags-import-issue"

provider = pulumi_aws.Provider(
    "aws",
    default_tags={
        "tags": {
            "Project": "x",
            "Environment": "dev",
            "CodeRepository": "infra",
        }
    } if test_import else None,
)
pulumi_aws.s3.BucketV2(
    bucket_name,
    bucket=bucket_name,
    tags={
        "Environment": "dev",
        "CodeRepository": "legacy-infra",
        "CreatedBy": "terraform",
    },
    opts=pulumi.ResourceOptions(
        import_=bucket_name if test_import else None,
        provider=provider,
        retain_on_delete=True,
    ),
)

Log output

$  pulumi up --skip-preview --diff --yes
Updating (dev)

View Live: https://app.pulumi.com/vertexinc/pulumi-defaulttags-import-issue/dev/updates/1

+ pulumi:pulumi:Stack: (create)
    [urn=urn:pulumi:dev::pulumi-defaulttags-import-issue::pulumi:pulumi:Stack::pulumi-defaulttags-import-issue-dev]
    + pulumi:providers:aws: (create)
        [urn=urn:pulumi:dev::pulumi-defaulttags-import-issue::pulumi:providers:aws::aws]
        region                   : "eu-west-1"
        skipCredentialsValidation: "false"
        skipRegionValidation     : "true"
        version                  : "6.38.1"
    + aws:s3/bucketV2:BucketV2: (create)
        [urn=urn:pulumi:dev::pulumi-defaulttags-import-issue::aws:s3/bucketV2:BucketV2::pulumi-aws-defaulttags-import-issue]
        [provider=urn:pulumi:dev::pulumi-defaulttags-import-issue::pulumi:providers:aws::aws::62cf8761-74c4-457a-8a89-8ca618d75c13]
        bucket      : "pulumi-aws-defaulttags-import-issue"
        forceDestroy: false
        tags        : {
            CodeRepository: "legacy-infra"
            CreatedBy     : "terraform"
            Environment   : "dev"
        }
        tagsAll     : {
            CodeRepository: "legacy-infra"
            CreatedBy     : "terraform"
            Environment   : "dev"
        }
        --outputs:--
        arn                               : "arn:aws:s3:::pulumi-aws-defaulttags-import-issue"
        bucketDomainName                  : "pulumi-aws-defaulttags-import-issue.s3.amazonaws.com"
        bucketRegionalDomainName          : "pulumi-aws-defaulttags-import-issue.s3.eu-west-1.amazonaws.com"
        grants                            : [
            [0]: {
                id         : "9a5403758502b3f38cc2f3a3f1bc8089ed04e2f0cd916cee1eaf1e9c8d6f408e"
                permissions: [
                    [0]: "FULL_CONTROL"
                ]
                type       : "CanonicalUser"
                uri        : ""
            }
        ]
        hostedZoneId                      : "Z1BKCTXD74EZPE"
        id                                : "pulumi-aws-defaulttags-import-issue"
        objectLockEnabled                 : false
        region                            : "eu-west-1"
        requestPayer                      : "BucketOwner"
        serverSideEncryptionConfigurations: [
            [0]: {
                rules: [
                    [0]: {
                        applyServerSideEncryptionByDefaults: [
                            [0]: {
                                kmsMasterKeyId: ""
                                sseAlgorithm  : "AES256"
                            }
                        ]
                        bucketKeyEnabled                   : false
                    }
                ]
            }
        ]
        versionings                       : [
            [0]: {
                enabled  : false
                mfaDelete: false
            }
        ]
Resources:
    + 3 created

Duration: 9s

$  pulumi state delete 'urn:pulumi:dev::pulumi-defaulttags-import-issue::aws:s3/bucketV2:BucketV2::pulumi-aws-defaulttags-import-issue'
 warning: This command will edit your stack's state directly. Confirm? Yes
Resource deleted

$  pulumi config set test_import 'true'

$  pulumi up --skip-preview --diff --yes
Updating (dev)

View Live: https://app.pulumi.com/vertexinc/pulumi-defaulttags-import-issue/dev/updates/3

  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:dev::pulumi-defaulttags-import-issue::pulumi:pulumi:Stack::pulumi-defaulttags-import-issue-dev]
    ~ pulumi:providers:aws: (update)
        [id=62cf8761-74c4-457a-8a89-8ca618d75c13]
        [urn=urn:pulumi:dev::pulumi-defaulttags-import-issue::pulumi:providers:aws::aws]
      + defaultTags: (json) {
          + tags: {
              + CodeRepository: "infra"
              + Environment   : "dev"
              + Project       : "x"
            }
        }

error: inputs to import do not match the existing resource
error: update failed
Resources:
    ~ 1 updated
    1 unchanged

Duration: 6s

Affected Resource(s)

No response

Output of pulumi about

➜  pulumi about                     
CLI          
Version      3.118.0
Go Version   go1.22.3
Go Compiler  gc

Plugins
KIND      NAME    VERSION
resource  aws     6.38.1
language  python  unknown

Host     
OS       darwin
Version  14.5
Arch     arm64

This project is written in python: executable='/private/tmp/pulumi-defaulttags-import-issue/venv/bin/python3' version='3.12.3'

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/piotr-bzdyl-vertexinc
User           piotr-bzdyl-vertexinc
Organizations  piotr-bzdyl-vertexinc, vertexinc
Token type     personal

Dependencies:
NAME        VERSION
pip         24.0
pulumi_aws  6.38.1

Pulumi locates its logs in /var/folders/pn/x8n_ksvj47128283fbf4fsqm0000gq/T/ by default
warning: Failed to get information about the current stack: No current stack

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[flostadler]

Thanks for reaching out @piotr-bzdyl-vertexinc! I'm sorry you're running into those issues. We're tracking a similar issue here: https://github.com/pulumi/pulumi-aws/issues/3311

I'm gonna have a look into this to see if I can find the cause or find a workaround.

[flostadler]

@piotr-bzdyl-vertexinc Sorry it seems like we're incorrectly handling the merge of default tags and regular tags during imports!

As a workaround you can try adding an ignore_changes statement for tags:

provider = pulumi_aws.Provider(
    "aws",
    default_tags={
        "tags": {
            "Project": "x",
            "Environment": "dev",
            "CodeRepository": "infra",
        }
    },
)

pulumi_aws.s3.BucketV2(
    bucket_name,
    bucket=bucket_name,
    tags={
        "Environment": "dev",
        "CodeRepository": "legacy-infra",
        "CreatedBy": "terraform",
    },
    opts=pulumi.ResourceOptions(
        import_=bucket_name if test_import else None,
        provider=provider,
        retain_on_delete=True,
        ignore_changes=["tags"],
    ),
)

With that change the import works for me locally. Without it fails with a diff in tags because they wrongly include the default tags as well:

warning: inputs to import do not match the existing resource; importing this resource will fail
    = aws:s3/bucketV2:BucketV2: (import)
        [id=pulumi-aws-defaulttags-import-issue]
        [urn=urn:pulumi:inline-policy-issue::inline-policy-issue::aws:s3/bucketV2:BucketV2::pulumi-aws-defaulttags-import-issue]
        [provider=urn:pulumi:inline-policy-issue::inline-policy-issue::pulumi:providers:aws::aws::04c25232-96ac-4d6e-bfb5-9d1d75e8ce2b]
      ~ tags: {
            CodeRepository: "legacy-infra"
            CreatedBy     : "terraform"
          + Environment   : "dev"
          + Project       : "x"
        }

[piotr-bzdyl-vertexinc]

That's strange as I did try the following workarounds before opening this issue (I'm sorry for not mentioning it earlier):

• adding pulumi.ResourceOptions.ignore_changes=["tags"]
• setting all tags from defaultTags that are not set on the imported resource to None hoping their presence in <resource>.tags will help

Also I double checked your suggestion with ignore_changes=["tags"] with my repro example and it also fails:

➜  pulumi up --skip-preview --diff --yes
Updating (dev)

View Live: https://app.pulumi.com/vertexinc/pulumi-defaulttags-import-issue/dev/updates/4

  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:dev::pulumi-defaulttags-import-issue::pulumi:pulumi:Stack::pulumi-defaulttags-import-issue-dev]
    ~ pulumi:providers:aws: (update)
        [id=1902f2e2-376a-4e67-bacc-fb343668a143]
        [urn=urn:pulumi:dev::pulumi-defaulttags-import-issue::pulumi:providers:aws::aws]
      + defaultTags: (json) {
          + tags: {
              + CodeRepository: "infra"
              + Environment   : "dev"
              + Project       : "x"
            }
        }

warning: Type checking failed:
warning: Unexpected type at field "tags.__defaults":
           expected string type, got [] type
warning: Type checking is still experimental. If you believe that a warning is incorrect,
please let us know by creating an issue at https://github.com/pulumi/pulumi-terraform-bridge/issues.
This will become a hard error in the future.
error: inputs to import do not match the existing resource
error: update failed
Resources:
    ~ 1 updated
    1 unchanged

Duration: 7s

[flostadler]

@piotr-bzdyl-vertexinc I see that you have the following provider config in your example:

provider = pulumi_aws.Provider(
    "aws",
    default_tags={
        "tags": {
            "Project": "x",
            "Environment": "dev",
            "CodeRepository": "infra",
        }
    } if test_import else None,
)

Can you try without the conditional default tags like so:

provider = pulumi_aws.Provider(
    "aws",
    default_tags={
        "tags": {
            "Project": "x",
            "Environment": "dev",
            "CodeRepository": "infra",
        }
    },
)

[t0yv0]

As a possible workaround would pulumi import command work? Unlike the import ResourceOption it does not run into inputs to import do not match the existing resource state.

[piotr-bzdyl-vertexinc]

As a possible workaround would pulumi import command work? Unlike the import ResourceOption it does not run into inputs to import do not match the existing resource state.

Yes, manual pulumi import works fine, but we prefer to run pulumi only from our CI jobs and this issue breaks our normal workflow.

[flostadler]

@piotr-bzdyl-vertexinc did the updated example from above without the conditional default tags work? I'm not able to reproduce it this way in combination with ignore_changes.

[piotr-bzdyl-vertexinc]

@piotr-bzdyl-vertexinc did the updated example from above without the conditional default tags work? I'm not able to reproduce it this way in combination with ignore_changes.

@flostadler the whole point of the conditional in the repro case was to create a resource with a different set of tags (as if it was created e.g. by Terraform) and then try to import it to a program that uses a different set of tags in the default tags configuration.

[flostadler]

@piotr-bzdyl-vertexinc Got it! In that case both the tags and tagsAll properties would need ignore_changes as a workaround.

I updated your code example to ignore changes to both tags and tagsAll and the import was successful after that:

Code example

import pulumi
import pulumi_aws

test_import = pulumi.Config().get_bool("test_import")

bucket_name = "defaulttags-import-issue-other"

provider = pulumi_aws.Provider(
  "aws",
  default_tags={
      "tags": {
          "Project": "x",
          "Environment": "dev",
          "CodeRepository": "infra",
      }
  } if test_import else None,
)
pulumi_aws.s3.BucketV2(
  bucket_name,
  bucket=bucket_name,
  tags={
      "Environment": "dev",
      "CodeRepository": "legacy-infra",
      "CreatedBy": "terraform",
  },
  opts=pulumi.ResourceOptions(
      import_=bucket_name if test_import else None,
      provider=provider,
      retain_on_delete=True,
      ignore_changes=["tags", "tagsAll"],
  ),
)

And here's the according test logs:

Test logs

➜  pulumi up --yes
Previewing update (flo-bucket-import-tags-issue)

View in Browser (Ctrl+O): https://app.pulumi.com/florian-pulumi-corp/4030-bucket-import/flo-bucket-import-tags-issue/previews/23e52b5e-41fb-4c02-a44f-10f7f939d40b

   Type                     Name                                             Plan
+   pulumi:pulumi:Stack      4030-bucket-import-flo-bucket-import-tags-issue  create
+   ├─ pulumi:providers:aws  aws                                              create
+   └─ aws:s3:BucketV2       defaulttags-import-issue-other                   create

Resources:
  + 3 to create

Updating (flo-bucket-import-tags-issue)

View in Browser (Ctrl+O): https://app.pulumi.com/florian-pulumi-corp/4030-bucket-import/flo-bucket-import-tags-issue/updates/8

   Type                     Name                                             Status
+   pulumi:pulumi:Stack      4030-bucket-import-flo-bucket-import-tags-issue  created (10s)
+   ├─ pulumi:providers:aws  aws                                              created (0.52s)
+   └─ aws:s3:BucketV2       defaulttags-import-issue-other                   created (5s)

Resources:
  + 3 created

Duration: 12s

➜  pulumi state delete "urn:pulumi:flo-bucket-import-tags-issue::4030-bucket-import::aws:s3/bucketV2:BucketV2::defaulttags-import-issue-other"
warning: This command will edit your stack's state directly. Confirm? Yes
Resource deleted
➜  pulumi config set test_import 'true'
➜  pulumi up --yes
Previewing update (flo-bucket-import-tags-issue)

View in Browser (Ctrl+O): https://app.pulumi.com/florian-pulumi-corp/4030-bucket-import/flo-bucket-import-tags-issue/previews/6975bf4f-1aef-4398-869c-c42b791b87c7

   Type                     Name                                             Plan       Info
   pulumi:pulumi:Stack      4030-bucket-import-flo-bucket-import-tags-issue
=   ├─ aws:s3:BucketV2       defaulttags-import-issue-other                   import     3 warnings
~   └─ pulumi:providers:aws  aws                                              update     [diff: +defaultTags]

Diagnostics:
aws:s3:BucketV2 (defaulttags-import-issue-other):
  warning: Type checking failed:
  warning: Unexpected type at field "tags.__defaults":
             expected string type, got [] type
  warning: Type checking is still experimental. If you believe that a warning is incorrect,
  please let us know by creating an issue at https://github.com/pulumi/pulumi-terraform-bridge/issues.
  This will become a hard error in the future.

Resources:
  ~ 1 to update
  = 1 to import
  2 changes. 1 unchanged

Updating (flo-bucket-import-tags-issue)

View in Browser (Ctrl+O): https://app.pulumi.com/florian-pulumi-corp/4030-bucket-import/flo-bucket-import-tags-issue/updates/10

   Type                     Name                                             Status              Info
   pulumi:pulumi:Stack      4030-bucket-import-flo-bucket-import-tags-issue
~   ├─ pulumi:providers:aws  aws                                              updated (0.53s)     [diff: +defaultTags]
=   └─ aws:s3:BucketV2       defaulttags-import-issue-other                   imported (4s)       3 warnings

Diagnostics:
aws:s3:BucketV2 (defaulttags-import-issue-other):
  warning: Type checking failed:
  warning: Unexpected type at field "tags.__defaults":
             expected string type, got [] type
  warning: Type checking is still experimental. If you believe that a warning is incorrect,
  please let us know by creating an issue at https://github.com/pulumi/pulumi-terraform-bridge/issues.
  This will become a hard error in the future.

Resources:
  ~ 1 updated
  = 1 imported
  2 changes. 1 unchanged

Duration: 9s

Does this solve your issue?

[piotr-bzdyl-vertexinc]

@flostadler Yes, indeed, adding both tags and tagsAll to ignore_changes helps and can be used as a workaround. I overlooked it and tried ignore_changes with tags only. Thanks!

[t0yv0]

Slight variation for readability:

import pulumi
import pulumi_aws

test_import = pulumi.Config().get_bool("test_import")

bucket_name = "pulumi-aws-defaulttags-import-issue-anton-6"

provider = pulumi_aws.Provider(
    "aws1",
    default_tags={
        "tags": {
            "Environment": "dev",
            "CodeRepository": "infra",
        }
    }
)

if not test_import:

    print('PROVISIONING')

    pulumi_aws.s3.BucketV2(
        bucket_name,
        bucket=bucket_name,
        tags={
            "CodeRepository": "legacy-infra",
            "CreatedBy": "terraform",
        },
        opts=pulumi.ResourceOptions(provider=provider))

else:

    print('IMPORTING')

    pulumi_aws.s3.BucketV2(
        bucket_name,
        bucket=bucket_name,
        tags={
            "CodeRepository": "legacy-infra",
            "CreatedBy": "terraform",
        },
        opts=pulumi.ResourceOptions(provider=provider, import_=bucket_name))

#!/usr/bin/env bash

set -euo pipefail

pulumi destroy --yes --skip-preview

pulumi stack export --file empty-state.json

pulumi config set test_import false
pulumi up --yes --skip-preview

pulumi stack import --file empty-state.json

sleep 5

pulumi config set test_import true
pulumi up --yes --skip-preview

This fails on latest but succeeds on pulumi-aws 5.42.0.

[pulumi-bot]

This issue has been addressed in PR #4169 and shipped in release v6.45.2.