Provider produced inconsistent result after apply with datadog_integration_aws and aws_cloudformation_stack

[andyshinn]

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version

Run terraform -v to show the version. If you are not running the latest version of Terraform, please upgrade because your issue may have already been fixed.

Terraform v1.0.2
on darwin_amd64
+ provider registry.terraform.io/datadog/datadog v3.4.0
+ provider registry.terraform.io/hashicorp/archive v2.2.0
+ provider registry.terraform.io/hashicorp/aws v3.63.0

Affected Resource(s)

Please list the resources as a list, for example:

• datadog_integration_aws
• datadog_integration_aws_log_collection
• datadog_integration_aws_lambda_arn
• aws_cloudformation_stack

If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.

Terraform Configuration Files

resource "aws_cloudformation_stack" "datadog_us_east_1" {
  name         = "datadog"
  capabilities = ["CAPABILITY_IAM", "CAPABILITY_NAMED_IAM", "CAPABILITY_AUTO_EXPAND"]
  parameters = {
    DdApiKey    = <my_api_key>
    ExternalId  = datadog_integration_aws.projects.external_id
    IAMRoleName = datadog_integration_aws.projects.role_name
  }
  template_url = "https://datadog-cloudformation-template.s3.amazonaws.com/aws/main.yaml"
}

resource "datadog_integration_aws" "projects" {
  account_id = data.aws_caller_identity.current.account_id
  role_name  = "DatadogIntegrationRole"
}

resource "datadog_integration_aws_log_collection" "log_collection" {
  account_id = data.aws_caller_identity.current.id
  services   = ["lambda"]
}

resource "datadog_integration_aws_lambda_arn" "lambda_arn_us_east_1" {
  account_id = data.aws_caller_identity.current.id
  lambda_arn = aws_cloudformation_stack.datadog_us_east_1.outputs.DatadogForwarderArn
}

Debug Output

Please provider a link to a GitHub Gist containing the complete debug output: https://www.terraform.io/docs/internals/debugging.html. Please do NOT paste the debug output in the issue; just paste a link to the Gist.

Panic Output

If Terraform produced a panic, please provide a link to a GitHub Gist containing the output of the crash.log.

Expected Behavior

Apply finishes without error.

Actual Behavior

Apply fails with error:

╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to datadog_integration_aws_log_collection.log_collection[0], provider "provider[\"registry.terraform.io/datadog/datadog\"]" produced an
│ unexpected new value: Root resource was present, but now absent.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply

Important Factoids

It works when applying again.

References

Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

• GH-1094

[therve]

Hi,

What are the versions of terraform and of the provider you're using? Thanks.

[andyshinn]

Sorry, I added that to the original description. Let me know if a debug output will be useful. I will need to tear it down to try again and grab debug output (as it is working after second try).

[collins-b]

I'm experiencing the same issue on the resource datadog_integration_aws_log_collection.

Error: Provider produced inconsistent result after apply
│
│ When applying changes to module.aws-integration.datadog_integration_aws_log_collection.main, provider
│ "module.aws-integration.provider[\"registry.terraform.io/datadog/datadog\"]" produced an unexpected new value: Root resource was present, but now absent.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵

[skarimo]

Hi, looks like the root cause is due to api returning 200 status code when there is an error. We have passed this on to the appropriate team and will follow up once we have an update. Thanks

[skarimo]

Following up here. To work around the issue, could you add a dependency between the log collection and lambda arn resource as the lambda arn is not being created before the log collection. For example:

resource "datadog_integration_aws_log_collection" "log_collection" {
  account_id = data.aws_caller_identity.current.id
  services   = ["lambda"]
  depends_on = [datadog_integration_aws_lambda_arn.lambda_arn_us_east_1]
}

resource "datadog_integration_aws_lambda_arn" "lambda_arn_us_east_1" {
  account_id = data.aws_caller_identity.current.id
  lambda_arn = aws_cloudformation_stack.datadog_us_east_1.outputs.DatadogForwarderArn
}

[collins-b]

Following up here. To work around the issue, could you add a dependency between the log collection and lambda arn resource as the lambda arn is not being created before the log collection. For example:

resource "datadog_integration_aws_log_collection" "log_collection" {
  account_id = data.aws_caller_identity.current.id
  services   = ["lambda"]
  depends_on = [datadog_integration_aws_lambda_arn.lambda_arn_us_east_1]
}

resource "datadog_integration_aws_lambda_arn" "lambda_arn_us_east_1" {
  account_id = data.aws_caller_identity.current.id
  lambda_arn = aws_cloudformation_stack.datadog_us_east_1.outputs.DatadogForwarderArn
}

This workaround worked for me.