terraform-provider-snowflake icon indicating copy to clipboard operation
terraform-provider-snowflake copied to clipboard

file_format error when i exec terraform apply a second time without any change

Open waterdo opened this issue 2 years ago • 4 comments

Provider Version

  • provider registry.terraform.io/chanzuckerberg/snowflake v0.25.28

Terraform Version

Terraform v1.1.7

Describe the bug

when I exec "terraform apply" a second time without any change of my code, return an error.

Plan: 0 to add, 1 to change, 0 to destroy. snowflake_file_format.external_storage: Modifying... [id=TF_DAF2|LOG|TFMYCSVFORMAT2] ╷ │ Error: error updating file format compression on TF_DAF2|LOG|TFMYCSVFORMAT2: 001008 (22023): SQL compilation error: │ invalid value [''] for parameter 'COMPRESSION' │ │ with snowflake_file_format.external_storage, │ on storagefileformat.tf line 3, in resource "snowflake_file_format" "external_storage": │ 3: resource "snowflake_file_format" "external_storage" { │ ╵ image

Expected behavior

Not return this error

Code samples and commands

resource "snowflake_file_format" "external_storage" { name = "TFMYCSVFORMAT2" database = "TF_DAF2" schema = "LOG" format_type = "csv" depends_on = [ snowflake_schema.schemaparatablas ] }

Additional context

Add any other context about the problem here.

waterdo avatar Mar 09 '22 09:03 waterdo

I am having the same issue. Has anyone found a workaround or a fix for this issue?

baugarcia avatar Jun 22 '22 11:06 baugarcia

Any work around this issue, we are having the same issue.

Kannedhara avatar Sep 28 '22 15:09 Kannedhara

@Kannedhara I fixed this for our environment by adding "compression = 'NONE'" in the file format terraform resource creation.

I am guessing the documentation is not really up to date and some parameters are mandatory for different format_types, so even if my parameter does not fix it for you, check the error and try to add a value for whatever parameter it complains.

baugarcia avatar Sep 28 '22 15:09 baugarcia

Here is a listing of the "required" fields (with what I believe are the default values) to get the file_format resource to work as expected:

  validate_utf8 = true
  compression = "AUTO"
  record_delimiter = "\n"
  binary_format = "HEX"
  escape = "NONE"
  time_format = "AUTO"
  timestamp_format = "AUTO"
  date_format = "AUTO"
  encoding = "UTF8"
  escape_unenclosed_field = "\\"
  field_optionally_enclosed_by = "NONE"

crowemi avatar Oct 07 '22 22:10 crowemi

This still seems to be occurring on 0.49.0 as of today. I'm having the same problem even passing default values as shown above.

icornett avatar Nov 02 '22 00:11 icornett

Same issue here.

jakobfloe avatar Nov 10 '22 08:11 jakobfloe

As @crowemi pointed out, the escape_unenclosed_field needs to be explicitly defined in the resource definition, otherwise it will fail (or will trigger a change every time).

...
  escape_unenclosed_field        = "\\"
...

This is the default setting, what works for me. Provider version 0.52

funes79 avatar Dec 19 '22 14:12 funes79

Issue still occurring at the moment - Correction by @crowemi worked like a charm

kstradinfs avatar Jan 06 '23 17:01 kstradinfs

Same issue here for creating a JSON file format. I solved by explicitly adding the default values. Note that @crowemi suggestion is for CSV.

aliceinnorway avatar Feb 23 '23 14:02 aliceinnorway

We are closing this issue as part of a cleanup described in announcement. If you believe that the issue is still valid in v0.89.0, please open a new ticket.

sfc-gh-asawicki avatar Apr 30 '24 16:04 sfc-gh-asawicki