Add "how to fix" instructions for "already exists" errors

[ghostsquad]

Hello!

• Vote on this issue by adding a 👍 reaction
• If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

When running pulumi up and getting an "already exists" error. It would be useful to also provide additional helpful messages for how to resolve such error. An example message may be:

To resolve this error, you can:

1. Change the name of your locally declared resource.
2. Import the resource.
3. Manually delete the external resource from the provider and rerun.

To Import, run this (though the variables here should actually be filled out, as pulumi already knows exactly what values should exist):

pulumi import [type] [name] [id] --parent [parent] --provider [provider]

### Affected area/feature

CLI

[lukehoban]

When running pulumi up and getting an "already exists" error

Can you give more details on what you mean by this?

I think perhaps you are referring to the error you get when you run pulumi import and try to import a resource that already exists in the state? In which case, we don't support that because it would overwrite your state. But we could as you say let you know to run pulumi state delete and then rerun the import. But perhaps you are talking about some other error?

Here's one case that I could imagine potentially improving the message on to help resolve:

➜  simples3forimport pulumi import -f import.json                                                                             
Previewing import (other)

View Live: https://app.pulumi.com/lukehoban/simples3forimport/other/previews/1e986bb9-eb71-49cc-8b77-1ccd735923ea

     Type                       Name                     Plan       Info
     pulumi:pulumi:Stack        simples3forimport-other             1 error
     └─ a:b:c                   foo                                 
 =      ├─ aws:s3:BucketObject  my-object                import     1 error
 =      └─ aws:s3:Bucket        my-bucket                import     1 error
 
Diagnostics:
  aws:s3:Bucket (my-bucket):
    error: Preview failed: resource 'urn:pulumi:other::simples3forimport::a:b:c$aws:s3/bucket:Bucket::my-bucket' already exists
 
  aws:s3:BucketObject (my-object):
    error: Preview failed: resource 'urn:pulumi:other::simples3forimport::a:b:c$aws:s3/bucketObject:BucketObject::my-object' already exists
 
  pulumi:pulumi:Stack (simples3forimport-other):
    error: preview failed

In this case, I already had my-bucket and my-object in my state, and trying to import over them raised this error. I could work around by pulumi state delete <urn> on each of them prior to the import.

[ghostsquad]

I mean when running Pulumi up, specifying a resource, with a name, or ID, etc that already exists. Pulumi will fail to create the resource because of the conflict. Leading me to usually either import or change the name of the locally declared resource.

[vorsprung]

This is the worst thing about pulumi. When i am developing infra and something fails in a odd way (as often happens when developing) pulumi cannot recover automatically from the situation and the commands to fix things are baroque. Or maybe they are very easy but there is no documentation.

I am a long time user of terraform and in this regard pulumi is very much worse

[Frassle]

I am a long time user of terraform and in this regard pulumi is very much worse

I'm confused by this because as far as I know terraform is exactly the same in this regard? If you try to create something that already exists both terraform and pulumi fail and just report back the error from the resource provider, which varies a little from provider to provider.

I'm sure we could do better here based on @ghostsquad's remarks, like if we had a way to determine the create failed because the provider considers it to already exist we could give an error message pointing at our import documentation which is probably what you want.

But from @vorsprung's comment it sounds like you think terraform is doing something else here? Can you give a concrete example of what we're doing worse than terraform here.

[Mankee]

this 409 issue is exactly why i'm switching back to terraform.

[Frassle]

Again, I'm confused because as I understand it Terraform is exactly the same in this regard!?

Both systems will error if you try to create something that already exists, you can't create a new S3 bucket called "foo", if that bucket already exists. In fact if anything Pulumi should be better here because by default we suffix names with random characters to avoid name collisions.

So what is Terraform doing here that's better than us? I don't think it's the idea in the first post on this issue, although I do think that's a good idea.

[sinfull1]

I get the same error when creating a user. the script fails and rest of the infra is not deployed. Need code pattern to catch this handle

aws:iam:User (log-user): error: 1 error occurred: * creating IAM User (log-user): EntityAlreadyExists: User with name log-user already exists. status code: 409, request id: 6462c4ca-3a1d-46fa-9cfe-b47f132280bd

[omidraha]

I have the same issue for some resource such as create Cloudflare record and create ecr user resources.

If two developers are working separately with Pulumi in local mode, and one of them has already created a resource such as a Cloudflare record or an ECR user, the other developer may encounter this error when running Pulumi up. This error occurs because the resource already exists and cannot be created again.

I think this error occurs because the second developer does not have that resource in their state.

I would like the Pulumi refresh command to synchronize the stack and update the state in order to prevent this error from occurring.

Diagnostics:
  pulumi:pulumi:Stack (devops-dev):
    error: update failed
    error: Resource monitor has terminated, shutting down

  aws:iam:User (ecr-user):
    error: 1 error occurred:
    	* creating IAM User (ecr-user): EntityAlreadyExists: User with name ecr-user already exists.
    	status code: 409, request id: ***

#3388

[Jaafar-Nasrallah]

I have the same issue in another context. A team mate has already created GCP cloudbuildv2 connection to github. upon deploying I get the Error creating Connection: Resource already exists. And I don't really know what to use for importing. figuring this out now

[lukehoban]

I think this error occurs because the second developer does not have that resource in their state.

I would like the Pulumi refresh command to synchronize the stack and update the state in order to prevent this error from occurring.

Can you share more on what you are imagining here? Is it that the two developers are working on the same stack, but without using the same statefile? How do they end up doing that? (Normal workflows with Pulumi Cloud state of S3 bucket state or similar wouldn't make this possible). Or is it that they are working on two different projects/stacks that just so happen to try to build the same (from the cloud providers perspective) thing?

getting an "already exists" error

I think the biggest challenge to directly addressing this issue is that there is no fixed notion of an "already exists" error. Today, the providers themselves (many of them bridged Terraform providers) decide whether to fail because a resource already exists, and decide what error message to bubble up. To address this in the ways suggested here, we'd need to have a way of identifying this class of errors, and adding additional context at the pulumi CLI layer about how to address. Definitely worth exploring how we could help more here!

[omidraha]

In cases where the development team is not using Pulumi Cloud and each developer login in locally with the command pulumi login --local, conflicts may arise when running pulumi up -y after making changes.

To prevent such conflicts, developers should follow these steps to synchronize their stack states :

1. Export the stack after making changes with pulumi up -y:

   pulumi stack export --file stack.json
   

2. Commit the changes and push them to the repository:

   git commit stack.json -m "Sync stack"
   

3. Retrieve the stack changes by importing the changes:

   pulumi stack import --file stack.json
   

By following these steps, developers ensure that they have the latest stack state and can avoid errors related to resource already exist during local development.

[JontyMC]

It would be useful to have a flag with pulumi up to say import the resource(s) if it exists. The situation for this is if the stack file is missing or corrupt and/or you want to start from clean stack. So you have the code and the resources exists but there is no stack. You shouldn't have to specify the id, it should be able to find by name.

[Frassle]

You shouldn't have to specify the id, it should be able to find by name.

Not all resources have names. It might be possible to fuzzy match by all the resources properties, but that doesn't feel like a safe default and could still require user input to distinguish between multiple resources with the same setup.

[JontyMC]

You shouldn't have to specify the id, it should be able to find by name.

Not all resources have names. It might be possible to fuzzy match by all the resources properties, but that doesn't feel like a safe default and could still require user input to distinguish between multiple resources with the same setup.

OK, but most do have names (at least in our stack), so an option to do it by name or fallback to ID would save a lot of time. We prefer to use our own suffix for resources names that is a short name for our environment {resourceName}-{env}, eg {resourceName}-staging.

[JontyMC]

Concrete example. We can work around this to a degree by doing the following:

var resourceGroup = new ResourceGroup("ResourceGroup", new()
{
    ResourceGroupName = resourceGroupName,
    Location = location
}, new()
{
    ImportId = importing
        ? (await GetResourceGroup.InvokeAsync(new() { ResourceGroupName = resourceGroupName })).Id
        : null
});

But this requires telling pulumi when you are importing and is a one off. It would be nice to do something like:

var resourceGroup = new ResourceGroup("ResourceGroup", new()
{
    ResourceGroupName = resourceGroupName,
    Location = location
}, new()
{
    AutoImportWhenNotInStackButDoesExist = true
});

Or even just be able to set that globally.

[akhilputhiry]

Same in Python.

import json
import pulumi
import pulumi_aws as aws


policy = aws.iam.Policy(
    "policy",
    name="my-test-policy",
    path="/",
    description="My test policy",
    policy=json.dumps(
        {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Sid": "ec2",
                    "Action": ["ec2:*"],
                    "Effect": "Allow",
                    "Resource": "*",
                }
            ],
        }
    ),
    opts=pulumi.ResourceOptions(import_="arn:aws:iam::123456789:policy/my-test-policy")
)

pulumi.export("policy_arn", policy.arn)