quickstart-examples icon indicating copy to clipboard operation
quickstart-examples copied to clipboard

Provide missing postgres variables on GCP bigquery pipeline

Open adatzer opened this issue 2 years ago • 2 comments

When setting up the default pipeline on GCP and running terraform plan with bigquery specific var-file, i.e.

terraform plan -var-file=bigquery.terraform.tfvars

there is error about missing postgres variables (Error: No value for required variable), namely for postgres_db_name, postgres_db_username, postgres_db_password. As those don't have defaults in variables.tf, a workaround was to provide dummy values for them in the bigquery tfvars file.

adatzer avatar Feb 06 '23 17:02 adatzer

I ran into the same issue in reverse -- when running terraform plan -var-file=postgres.terraform.tfvars I get: Error: Error in function call │ │ on main.tf line 234, in locals: │ 234: bq_loader_dead_letter_bucket_name = coalesce( │ 235: join("", google_storage_bucket.bq_loader_dead_letter_bucket.*.name), │ 236: var.bigquery_loader_dead_letter_bucket_name, │ 237: ) │ ├──────────────── │ │ while calling coalesce(vals...) │ │ google_storage_bucket.bq_loader_dead_letter_bucket is empty tuple │ │ var.bigquery_loader_dead_letter_bucket_name is "" │ │ Call to function "coalesce" failed: no non-null, non-empty-string arguments. Even though I'm not using bigquery. adding the var with a dummy value worked for me as well

lenmazzone avatar Jul 31 '23 16:07 lenmazzone

Hey @lenmazzone thanks for raising this - we should have a fix for this coming very soon + updates to all of the underlying GCP modules to remove deprecated functions. Watch this space!

jbeemster avatar Jul 31 '23 22:07 jbeemster