[Bug]: Docker cache doesn't work?

[TheM1Stery]

Description

My app is a simple rust web server(axum). My Dockerfile utilizes docker caching(via cargo-chef) to reduce compilation times of rust(which are notoriously slow). After each deployment the images used inside the dockerfile are downloaded again and the compilation begins anew(making cargo-chef useless).

I also checked the cache by doing back-to-back(2 min diff) commits(like changing stuff in strings lol) and to my surprise docker used the cache.

Does this have to do anything with Automated Cleanups(https://coolify.io/docs/knowledge-base/server/automated-cleanup)? Probably not, cause my disk space is plenty for now

Docker cache works locally on my machine

Minimal Reproduction (if possible, example repository)

This is the Dockerfile that i was using:

FROM rust:1.76 as toolchain
RUN cargo install cargo-chef --locked
RUN cargo install sqlx-cli --no-default-features --features sqlite
WORKDIR /app

FROM toolchain as planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json


FROM toolchain as builder
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json
COPY . .
# check query macros
ENV DATABASE_URL=sqlite:db.sqlite
RUN sqlx database create
RUN sqlx migrate run
RUN cargo build --release

FROM debian:bookworm-slim as runtime
WORKDIR /app
ENV SERVER_ADDRESS=0.0.0.0
ENV SERVER_PORT=8000
COPY --from=builder /app/target/release/axum-web-test /app/axum-web-test
EXPOSE 8000
ENTRYPOINT [ "/app/axum-web-test" ]

I probably could reduce the compilation time by removing cargo-chef and sqlx-cli and by creating a docker image that already has both of them but i don't that's the best solution? It would render cargo-chef useless for this usecase if the image's cache is wiped?

Exception or Error

Docker cache doesn't work, the image is recreated from scratch after each commit making the build time large

Version

v4.0.0-beta.258

[TheM1Stery]

FROM rust:1.76 as builder
# check query macros
ENV DATABASE_URL=sqlite:db.sqlite
RUN --mount=type=cache,target=/usr/local/cargo/registry cargo install sqlx-cli --no-default-features --features sqlite
WORKDIR /app
COPY . .
RUN sqlx database create
RUN sqlx migrate run
RUN --mount=type=cache,target=/usr/local/cargo/registry --mount=type=cache,target=/app/target cargo build --release && \
    mv /app/target/release/axum-web-test /app


FROM debian:bookworm-slim as runtime
WORKDIR /app
ENV SERVER_ADDRESS=0.0.0.0
ENV SERVER_PORT=8000
COPY --from=builder /app/axum-web-test /app
EXPOSE 8000
ENTRYPOINT [ "/app/axum-web-test" ]

I tried to use --mount=type=cache instead of cargo-chef but the builds are still not cached. Everything works locally.

[andrasbacsai]

I tried to replicate, but I could not.

Can you please make an example public repository (with axum + sqlite)?

[TheM1Stery]

I tried to replicate, but I could not.

Can you please make an example public repository (with axum + sqlite)?

Hello! Thanks for looking into this issue.

Here is the repository: https://github.com/TheM1Stery/rust-web-site

As you can see from these deployments: the build time is super long.

Looking at the logs, i can see from the logs the images used inside the dockerfile are downloaded every time? And the compilation starts from scratch everytime: coolify-logs.txt Sorry for bad formatting in logs(it copied it like that lol)

[andrasbacsai]

It should be something on your server, as on my side, it uses cache.

The cleanup process checks the disk usage every two minutes, and if the threshold is above the set limit for the server, it cleans up the build cache.

[TheM1Stery]

It should be something on your server, as on my side, it uses cache.

The cleanup process checks the disk usage every two minutes, and if the threshold is above the set limit for the server, it cleans up the build cache.

Strange, i even created a new vps from Hetzner using different OSes(Debian and Ubuntu 22.04) and still had the same issue. I'll try with a new server again then. Still, thanks for looking into this issue.

The cache only worked for me when i did the commit minutes after the last one.

[TheM1Stery]

I checked again. The cache only works minutes after the last build. After, i push another commit, the image is built from scratch again. I'll test this again in another new server in Hetzner, but i don't think it will make a difference

edit: by not being cached in the screenshots i meant that the cache is not being used

[TheM1Stery]

I tried with a brand new server(Fedora 40 this time), the same issue persists. New deployments don't use the cache from previous deployments. The rust and debian images used inside the dockerfile are downloaded again and the compilation begins from scratch. Sorry for bothering

[crisog]

I tried with a brand new server(Fedora 40 this time), the same issue persists. New deployments don't use the cache from previous deployments. The rust and debian images used inside the dockerfile are downloaded again and the compilation begins from scratch. Sorry for bothering

I've been having the same problem, within 2-3 minutes cache works perfectly. After longer than that, it starts from scratch. v4.0.0-beta.276.

[algora-pbc[bot]]

💎 $50 bounty • CrisOG

Steps to solve:

1. Start working: Comment /attempt #2003 with your implementation plan
2. Submit work: Create a pull request including /claim #2003 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to coollabsio/coolify!

Add a bounty • Share on socials

[AlejandroAkbal]

My deployments have never used cache either Git + docker compose

[TheM1Stery]

It looks like caching works now? I do not know what caused the fix but it works for me right now. Will monitor it further but if it continues to work, this issue can be closed.

[TheM1Stery]

It seems to work for me, even after a lot of commits. I'm closing the issue.

Thanks for Coolify @andrasbacsai !