[bug] datadog_gcp_integration not working properly / incorrect documentation

[sgringwe]

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version

0.12.18

Affected Resource(s)

• datadog_integration_gcp

Terraform Configuration Files

resource "google_service_account" "datadog_insights_monitoring_account" {
  account_id   = "datadog-insights-monitoring"
  display_name = "Datadog Monitoring"
  description  = "A Service Account that is configured into Datadog to support GCP monitoring integrations"
}

resource "google_project_iam_member" "datadog_monitoring_account_metrics_viewer" {
  project = var.project
  role    = "roles/monitoring.viewer"
  member  = "serviceAccount:${google_service_account.datadog_insights_monitoring_account.email}"
}

resource "google_project_iam_member" "datadog_daemon_set_compute_instance_getter" {
  project = var.project
  role    = "roles/compute.viewer"
  member  = "serviceAccount:${google_service_account.datadog_insights_monitoring_account.email}"
}

resource "google_project_iam_member" "datadog_monitoring_account_cloud_asset_viewer" {
  project = var.project
  role    = "roles/cloudasset.viewer"
  member  = "serviceAccount:${google_service_account.datadog_insights_monitoring_account.email}"
}

## GCP/Datadog integration
resource "google_service_account_key" "datadog_insights_monitoring" {
  service_account_id = google_service_account.datadog_insights_monitoring_account.name
  public_key_type    = "TYPE_X509_PEM_FILE"
}

resource "datadog_integration_gcp" "gcp_project_integration" {
  project_id = var.project

  private_key_id = jsondecode(base64decode(google_service_account_key.datadog_insights_monitoring.private_key))["private_key_id"]
  private_key    = jsondecode(base64decode(google_service_account_key.datadog_insights_monitoring.private_key))["private_key"]

  client_email = google_service_account.datadog_insights_monitoring_account.email
  client_id    = google_service_account.datadog_insights_monitoring_account.unique_id
}

Debug Output

Please provider a link to a GitHub Gist containing the complete debug output: https://www.terraform.io/docs/internals/debugging.html. Please do NOT paste the debug output in the issue; just paste a link to the Gist.

The apply "succeeds".

Panic Output

If Terraform produced a panic, please provide a link to a GitHub Gist containing the output of the crash.log.

Expected Behavior

What should have happened?

Integration is working

Actual Behavior

What actually happened?

Integration is not working. Integrations UI shows error message "- Datadog could not query this project properly. You need to reconfigure this project's credentials/permissions"

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply

Important Factoids

n/a

References

n/a

[jirikuncar]

@sgringwe can you please open a support ticket or provide more details? Thank you!

[sgringwe]

@jirikuncar what additional details would you like to see?

[jirikuncar]

It's a bit difficult for me to debug the problem without the request that was made. We can help you through the support ticket to investigate the real cause of this problem and if it happens to be terraform-provider-datadog we will have more details then. Thank you for your understanding.

[jskowalski]

I'm having the same issue. Complete the google service account, upload file (success) then errors with "- Datadog could not query this project properly. You need to reconfigure this project's credentials/permissions"

[kassim]

I'm experiencing the same issue/error warning when trying to set up the datadog gcp integration. Is there a way to debug or solve?

[davidtom]

Encountering this issue as well, which is blocking my team from adopting Datadog. Would love to have a solution for this.

[kassim]

Actually I came back to it the day after and it seemed to be working.

Could be GCP took a while to make permissions work.. Or maybe someone in my huge organisation fixed something with our config and I wasn't aware.

On Thu, 24 Dec 2020, 17:42 David Tomczyk, [email protected] wrote:

Encountering this issue as well, which is blocking my team from adopting Datadog. Would love to have a solution for this.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/DataDog/terraform-provider-datadog/issues/464#issuecomment-750924647, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUCP2DK4S66P6AINSHYHULSWNVNVANCNFSM4LW2ZN2A .

[davidtom]

Actually I came back to it the day after and it seemed to be working. Could be GCP took a while to make permissions work.. Or maybe someone in my huge organisation fixed something with our config and I wasn't aware.

Interesting. Was wondering if that was the issue, but waiting for the permissions didn't seem to change anything for us.

However, I was able to get it working using the following roles for the service account:

resource "google_project_iam_member" "service_account" {
  for_each = toset([
    "roles/compute.viewer",
    "roles/container.viewer",
    "roles/monitoring.viewer",
  ])

  project = var.project
  role    = each.value
  member  = "serviceAccount:${google_service_account.datadog.email}"
}

I got those from this module.

Hopefully this can help someone, but very curious why this works even though its quite different than what is in the documentation.

[nmuesch]

Hey, apologies for the delay here. I wanted to check in and see if you were still experiencing an issue with this. Would you also mind pointing to the documentation you're referring to?

Looking at the documentation here - https://docs.datadoghq.com/integrations/google_cloud_platform/?tab=datadogussite#installation the list of roles seem to be present.

[markus-wa]

We're seeing a similar issue. On Friday our GCP integration stopped working, seemingly out of the blue. Monday we rotated the associated GCP SA Key and updated the Terraform resource gcp_project_integration which was successful - but this didn't help.

the service account has the appropriate permissions

[jtafurth]

I'm also having the same problem

Stopped working last week, I updated the provider to the latest version but still doesn't work, I do get data back from google but it seems to be working intermittently.

[xavidop]

Hi we are facing the same issue! I just followed this configuration: https://github.com/nephosolutions/terraform-google-datadog-integration/blob/master/main.tf#L17

and using the terraform resource its not working, in fact, when I go to that service account and I export the json file manually and then I import it on datatadog manually as well, it works.

Really really weird, I am using the datadog provider 3.4.0

[sergio-guillen-exa]

I am also having the same error but not only with the Datadog Terraform provider. What I tested so far without success:

• GCP project with billing, compute, monitoring, cloud asset services enabled
• Service account that has viewer roles for compute, monitoring and cloud asset (one service account created with Terraform and the other created manually)
• Credentials created with terraform and manually
• Datadog integration keeps having the same error using Terraform resource, manually creating using Terraform service account and manually creating using the manually-created service account.

The versions I used for the provider are 3.4.0 and 3.2.0 (this was using for other projects)

Is it possible that the GCP project requires another service to be enabled?

[ajmunoz411]

I'm also having this same issue and haven't yet been able to find a fix. I've been using the GCP integration for over 10 months without issue but today started having this issue. It was right after I tried adding a host_filter that the integration started showing up as Broken with the Datadog could not query this project properly. You need to reconfigure this project's credentials/permissions error. So far, I've tried:

• creating the service account (with the viewer roles for compute, monitoring, and cloud asset) both manually and via terraform
• tried the alternate roles suggested by @davidtom above
• I'm on on version 3.12.0 for the DataDog provider

Still no luck so far, wondering if there are any other things to try to resolve this?

[c-p-b]

I was able to get this working manually, basically by removing any automations created by terraform and then manually creating a key in GCP UI, then manually adding that json file into Datadog UI. I had terraform create the service account with these permissions:

resource "google_project_iam_member" "service_account" {
  for_each = toset([
    "roles/compute.viewer",
    "roles/cloudasset.viewer",
    "roles/monitoring.viewer",
  ])

  project = var.project
  role    = each.value
  member  = "serviceAccount:${google_service_account.datadog.email}"
}

[chruzcruz]

Same issue here on 3.24.0

resource "google_project_service" "enable_apis" {
  for_each = toset([
    "pubsub.googleapis.com",
    "cloudbilling.googleapis.com",
    "monitoring.googleapis.com",
    "compute.googleapis.com",
    "cloudasset.googleapis.com",
  ])
  project            = local.project_id
  service            = each.value
  disable_on_destroy = false
  
  resource "google_service_account" "datadog" {
  project = local.project_id
  account_id   = "datadog-integration-account"
  display_name = "Datadog Integration with ${local.project_id}"
}

resource "google_service_account_key" "datadog" {
  service_account_id = google_service_account.datadog.name
}
  
  resource "google_project_iam_member" "service_account" {
  for_each = toset([
    "roles/compute.viewer",
    "roles/container.viewer",
    "roles/monitoring.viewer",
    "roles/cloudasset.viewer",
  ])

  project = local.project_id
  role    = each.value
  member  = "serviceAccount:${google_service_account.datadog.email}"
}

}

resource "datadog_integration_gcp" "dd_gcp_project_integration" {
  project_id     = jsondecode(base64decode(google_service_account_key.datadog.private_key))["project_id"]
  private_key    = jsondecode(base64decode(google_service_account_key.datadog.private_key))["private_key"]
  private_key_id = jsondecode(base64decode(google_service_account_key.datadog.private_key))["private_key_id"]
  client_email   = jsondecode(base64decode(google_service_account_key.datadog.private_key))["client_email"]
  client_id      = jsondecode(base64decode(google_service_account_key.datadog.private_key))["client_id"]
  host_filters   = join(",", sort(var.host_filters))

}

Output

- Datadog could not query this project properly. You need to reconfigure this project's credentials/permissions

[nkzou]

The cause of this issue is not due to the Terraform Provider, but rather some underlying configuration error with the Datadog integration and/or GCP. That specific error message is a catch-all, with a variety of possible misconfigurations - we cannot help debug this in a Github issue. Please open a ticket with Datadog support at https://help.datadoghq.com/, and a support engineer will work with you to resolve the issue with much more visibility than we can here.