padok-team
Cache terraform, terragrunt, tofu binaries
Hello, Burrito is currently caching providers into Hermitcrab, which is nice. It could be nice to cache the binaries as well to avoid hundreds/thousands of downloads. I think this would be better for scalability.
I can see tenv supports artifactory. While this is a good tool I have at my company, I think not all companies can afford it 😄 I was thinking we could maybe mimic this with a cache like varnish.
Would you like an opt-in cache into Burrito chart (like hermitcrab) or just have a documentation explaining how to use your own cache?
Maybe the datastore could be used for that as well but I guess this requires more work
Hi!
What I would suggest, instead of a remote cache, is to embed your binaries (multiple versions allowed!) in the Docker image of the runner. And then use in the overrideRunnerSpec field on a TerraformRepo/TerraformLayer or globally in runner.image in Burrito's config.
tenv should detect there are already present.
Quick example snippet:
FROM alpine:latest AS download
ARG TARGETARCH
RUN mkdir -p /runner/bin
RUN curl -fsSL https://releases.hashicorp.com/terraform/1.4.6/terraform_1.4.6_linux_${TARGETARCH}.zip -o /tmp/terraform.zip && \
unzip /tmp/terraform.zip -d /runner/bin && \
mv /runner/bin/terraform /runner/bin/terraform-1.4.6 && \
curl -fsSL https://releases.hashicorp.com/terraform/1.4.7/terraform_1.4.7_linux_${TARGETARCH}.zip -o /tmp/terraform.zip && \
unzip /tmp/terraform.zip -d /runner/bin && \
mv /runner/bin/terraform /runner/bin/terraform-1.4.7
RUN curl -fsSL https://github.com/gruntwork-io/terragrunt/releases/download/v0.47.0/terragrunt_linux_${TARGETARCH} -o /runner/bin/terragrunt-0.47.0 && \
curl -fsSL https://github.com/gruntwork-io/terragrunt/releases/download/v0.48.7/terragrunt_linux_${TARGETARCH} -o /runner/bin/terragrunt-0.48.7
FROM ghcr.io/padok-team/burrito:v0.6.5
USER root
COPY --from=download /runner/bin/ /runner/bin/
RUN chown -R burrito:burrito /runner/bin
USER burrito
Indeed I forgot this possibility. However, depending on layers age and teams (org-wide), we may have tens of combinations of terraform/terragrunt. Also image maintenance would be quite hard, especially with terragrunt. If your version isn't in the image, this can silently result in higher egress costs depending on your tf activity. That might be worth it for big infras 🙂
In fact, we need to simplify our versions now we have burrito regular plans. I think we'll move to a global .terraform-version and .terragrunt-version per repo, configure renovatebot to follow them. If it creates a PR, we'll know which layer fails thanks to https://docs.burrito.tf/user-guide/additionnal-trigger-path/ on both files for all layers. This will drastically reduce versions count in the whole infra so we can have an image for it 👍
This is something we had in mind too, but that's something Burrito cannot handle on its own. However I think documenting different patterns that could be used to cache those binaries, like a ReadWriteMany volume containing all binaries and mounted on the runner pods could drastically reduce egress costs.
@ricardoleal I'm not that familiar with oras, could you elaborate on how it could help ? Really interested if there's a solution out there we're not seeing 🚀
Ok a simple example can be found here
You can see this simple example:
# Step 1: Create a sample file
echo "hello world" > artifact.txt
# Step 2: Push an artifact
oras push --plain-http localhost:5000/hello-artifact:v1 \
--artifact-type application/vnd.acme.rocket.config \
artifact.txt:text/plain
Uploading a948904f2f0f artifact.txt
Uploaded a948904f2f0f artifact.txt
Pushed [registry] localhost:5000/hello-artifact:v1
Digest: sha256:bcdd6799fed0fca0eaedfc1c642f3d1dd7b8e78b43986a89935d6fe217a09cee
Mount OCI image as volume in a pod.
Example:
…
kind: Pod
spec:
containers:
- …
volumeMounts:
- name: my-volume
mountPath: /path/to/directory
volumes:
- name: my-volume
image:
reference: hello-artifact:v1
you can see also here : https://kubernetes.io/blog/2024/08/16/kubernetes-1-31-image-volume-source/#example
I can't make a full demo for this now, but in a nutshell would be to add all files to that OCI image and mount them as volume.
If some new image is needed , tenv should be able to install during the runtime.
@ricardoleal Thanks for the tips didn't think it was possible to mount an oci image as a volume in a pod, that seems like a good pattern we could document for that purpose
For info, I managed to create a rather simple Dockerfile that downloads Terraform/OpenTofu/Terragrunt versions for Burrito to use them:
FROM ghcr.io/tofuutils/tenv:latest AS download
RUN mkdir -p /runner/bin
ENV TENV_ROOT=/runner/bin
RUN tenv tf install "~>1.13.0"
RUN tenv tg install "~>0.92.0"
FROM ghcr.io/padok-team/burrito:v0.8.1 AS burrito
USER root
COPY --from=download /runner/bin/ /runner/bin/
RUN chown -R burrito:burrito /runner/bin
USER burrito
tenv is the tool Burrito uses to automatically downloads binaries so using their image ensures that the /runner/bin folder structure is well formed.
To check if it works you should see this in Burrito runner's logs:
time="2025-10-30T17:13:52Z" level=info msg="installing binaries..."
time="2025-10-30T17:13:52Z" level=info msg="found compatible terraform version 1.13.4 already installed"
time="2025-10-30T17:13:52Z" level=info msg="using terraform version 1.13.4"
time="2025-10-30T17:13:52Z" level=info msg="found compatible terragrunt version 0.92.1 already installed"
time="2025-10-30T17:13:52Z" level=info msg="using Terragrunt version 0.92.1 as wrapper for terraform"