Feature request: Better way to provide package for compute services

[fruwe]

Feature request to provide the Compute@Edge binary package by other means than a file name.

When deploying a compute@edge service one has to specify a file name in the fastly_service_compute > package > filename setting. Especially when deploying a service using Terraform Cloud triggered by version control this becomes problematic, because it forces one to commit the built binary package to git.

Instead of committing a 5MB or so file on each change, I would kindly request a new feature to improve this. Possible solutions could be

1. Provide the relative path to the source code in the package setting and then auto build from that.

resource "fastly_service_compute" "test" {
  ...
  package {
    source = "./src/"
  }
}

2. Set content instead of filename.

resource "fastly_service_compute" "test" {
  ...
  package {
    content = file("./package.tar.gz")
  }
}

This could help, because we could provide the file content by other means, e.g. input variables

resource "fastly_service_compute" "test" {
  ...
  package {
    content = base64decode(var.package_content)
  }
}

3. Allow URLs in filename. Build and upload the image to the cloud before terraform apply and then download the file from the specified URL

resource "fastly_service_compute" "test" {
  ...
  package {
    filename = "https://storage.googleapis.com/xxx/package.tar.gz"
  }
}

[Integralist]

Thanks @fruwe for these suggestions.

I agree that it would be good to offer more flexibility here.

Unfortunately, we can't give exact timelines for feature requests but we'll update this ticket once we've had a chance to consider the possible options.

[fruwe]

Thank you!

[Integralist]

👋🏻 I just wanted to circle back and update this ticket with more information.

We wouldn't want to do the first option because Terraform should not be responsible for building a user's package from source code.

I started to investigate the third option for supporting a remote URL, but discovered this wouldn't be viable because we still need to provide source_code_hash in the user's Terraform configuration in order for the Terraform provider to be able to identify whether there was a change to the package that would require an update.

For example, if the user is trying to reference a package that is external (i.e. the package is not on the user's local disk), then they're not going to be able to set source_code_hash using Terraform's filesha512() function.

I then spent some time investigating whether we could internalise the source_code_hash attribute, so that a user wouldn't have to specify it manually but this wasn't possible either due to limitations in the Terraform processing model.

Specifically, if we dynamically set source_code_hash into the state as part of a READ operation (i.e. state refresh), then although we can temporarily update the state to the changed hash, it doesn't cause Terraform to trigger an UPDATE because it requires the attribute in the state to be compared against something in the user's Terraform configuration (and also the updated state is then dropped by Terraform because of this).

The second option, a "content" attribute, should be possible (using filebase64()) because the contents of the file is always defined in the user's Terraform configuration, and subsequently will be directly comparable to the Terraform state (no need for source_code_hash to be set to identify changes).

One caveat to the second option, and source_code_hash in general as we move forward, is that the Fastly API will (at some point in the future) switch from returning a hashsum of the package.tar.gz to a hashsum of the main.wasm and fastly.toml files (this is to avoid issues where the package.tar.gz had no logical changes but a change in metadata would cause the package to look like it had been updated, resulting in an unnecessary package upload).

[fruwe]

Thanks for looking into this and the update!

With option 1 and 3 being not feasible, option 2 still seems best, even though the deployment rate would be higher than necessary.

Maybe another option could be to have the option to disable deployments of new code via terraform (but enable the other settings such as domains), and then deploy the code with Github actions, could it be? I assume that would not work when deploying the first time, would it?

[glerchundi]

Hi all, just wanted to show you how we're doing it right now on our side and confirm that the best option would be the second one.

We're using GCS bucket as our WASM packages registry and we wanted to directly use fastly_service_compute without needing to write the file to the local fs first.

Currently we're doing this:

data "google_storage_bucket_object_content" "this" {
  name   = "component/package.tar.gz.b64"
  bucket = "my-fastly-repository"
}

resource "local_file" "this" {
  content_base64 = data.google_storage_bucket_object_content.key.content
  filename       = "package.tar.gz"
}

resource "fastly_service_compute" "service_compute" {
  name = "Service compute"
  package {
    filename         = "package.tar.gz"
    source_code_hash = filesha512("package.tar.gz")
  }
}

And it works but...in an ideal world I would prefer to avoid the local file saving and directly pipe to the resource:

data "google_storage_bucket_object_content" "this" {
  name   = "component/package.tar.gz.b64"
  bucket = "my-fastly-repository"
}

resource "fastly_service_compute" "service_compute" {
  name = "Service compute"
  package {
    content          = base64decode(data.google_storage_bucket_object_content.key.content)
    source_code_hash = sha512(base64decode(data.google_storage_bucket_object_content.key.content))
  }
}

WDYT?

[glerchundi]

I submitted a new PR to fastly/go-fastly#384 that would unlock the possibility to further continue with this proposal. Any feedback/review there would be really appreciated! 🙏

[Integralist]

Hi @glerchundi

I submitted a new PR to https://github.com/fastly/go-fastly/pull/384 that would unlock the possibility to further continue with this proposal.

Great! 🙂

I'll aim to take a look at that next week and provide feedback. Thanks!

[glerchundi]

Thanks! 🙏

[Integralist]

👋🏻 @glerchundi apologies for the delay but I've had other priorities since we last spoke and this has been my first opportunity to revisit this issue.

I just wanted to bring the following 'draft' PR to your attention. I've not tested it at all, but I'm on PTO for a few days and just wanted to let you know that I'm investigating option 2 described in this issue.

https://github.com/fastly/terraform-provider-fastly/pull/661

[Integralist]

I've done further testing and have marked my PR ready for review: https://github.com/fastly/terraform-provider-fastly/pull/661

Please take a look and let me know your thoughts and if this is meeting your needs.

Thanks.

[Integralist]

Specifically, see my comment as to why we need the content attribute to be a base64 encoded string: https://github.com/fastly/terraform-provider-fastly/pull/661#pullrequestreview-1339970356

[fruwe]

Awesome, thanks!!