Store source and output file hashes instead of actual file data in tfstate

[neechbear]

Terraform Version

$ terraform -v
Terraform v0.11.7
+ provider.archive v1.0.3
+ provider.aws (unversioned)

Affected Resource(s)

• archive_file

Terraform Configuration Files

provider "aws" { }

data "archive_file" "myarchive" {
  type        = "zip"
  output_path = "myarchive.zip"

  source {
    content  = "${file("bigfile")}"
    filename = "bigfile.1"
  }
  source {
    content  = "${file("bigfile")}"
    filename = "bigfile.2"
  }
}

Debug Output

• https://gist.github.com/neechbear/db93fbd95a1fa333dbb607b061aeac46
• https://gist.github.com/neechbear/f8e9bf69d920ff4991a4c6fea2c90d2b

Panic Output

N/A

Expected Behavior

It would be preferable to only store a hash of the source and output file contents in the Terraform state file, instead of the entire file contents.

Actual Behavior

The entire source file contents are stored in the Terraform state file, which needs to be JSON encoded. Thus creating a ZIP file with two 100MB sparse files inside will result in a Terraform state file of 1.2GB in size! As you can imagine, this is very slow, especially if you store your state in an S3 bucket.

Steps to Reproduce

Create an archive containing one or more large files using archive_file.

Important Factoids

An example of this in action:

• https://gist.github.com/neechbear/f8e9bf69d920ff4991a4c6fea2c90d2b

References

No other references that I could find.

For my own reference, my internal bug tracking number is GS-5487.