enhance: Improve the docker build speed

[TreyWW]

At the moment, our docker build . for me takes around ~4-5 mins on average. It uses no caching so even when changing 0 files the build takes just as long. Here's some of my build times:

7m 11s, 5m 31s, 5m 15s, 4m 49s and since a new Dockerfile test: 8m 46s, 10m 36s, 12m 1s, 2m 7s, 5m 15s

So both aren't doing so well, our current set is more consistent, but caching should be in place and builds should not take anywhere near this long.

CURRENT DOCKERFILE

FROM python:3.12-alpine

RUN pip install poetry==1.7.1

ENV POETRY_NO_INTERACTION=1 \
    POETRY_VIRTUALENVS_IN_PROJECT=1 \
    POETRY_VIRTUALENVS_CREATE=false \
    POETRY_CACHE_DIR=/tmp/poetry_cache

WORKDIR /MyFinances

COPY . .

#RUN #apk --no-cache --update add \
#    mariadb-connector-c-dev \
#    py-pip \
#    musl-dev \
#    gcc \
#    mariadb-dev \
#    libffi-dev


# Install build dependencies
RUN apk add --virtual .build-deps py-pip musl-dev gcc

# Install MySQL dependencies and packages if DATABASE_TYPE is mysql
RUN if [ "${DATABASE_TYPE}" = "mysql" ]; then \
        apk add --no-cache mariadb-dev && \
        poetry install --only mysql; \
    fi

# Install PostgreSQL dependencies and packages if TESTING is not true or DATABASE_TYPE is postgres
RUN if [ "${TESTING}" != "true" ] || [ "${DATABASE_TYPE}" = "postgres" ]; then \
        apk add --no-cache postgresql-dev && \
        poetry install --only postgres; \
    fi

# Clean up build dependencies
RUN apk del .build-deps

RUN poetry install --without dev,mysql,postgres --no-root && rm -rf $POETRY_CACHE_DIR

RUN chmod +x infrastructure/backend/scripts/*
RUN chmod +x infrastructure/backend/scripts/tests/*
ENTRYPOINT ["sh", "infrastructure/backend/scripts/entrypoint.sh"]

EXPOSE 10012
EXPOSE 9012

Now I don't know much about docker files, never really prioritised build speeds as I don't often build much, but since deploying this app more it has come clear that this is too long for what it should be - right?

Any help on how we can improve this, add better caching, excluding certain files that may not be needed, faster docker pushes, anything like that just leave a comment. Much appreciated :)

EDIT: Removed "new dockerfile" as it doesn't actually deploy properly. Maybe I should have tested it rather than just going off the build speed ;) Nevertheless this issue still needs to be solved

[TreyWW]

Here's 10 of my last builds. Not sure if this is just my computer or if it's consistent throughout the project

[Domejko]

Hey @TreyWW

I did some test and I assume that there might be something on your side. Every initial build on Linux took me between 40-55 seconds, on Windows 10 it was 60-80 seconds and every subsequent build takes 1s or less. The most time in initial builds takes installation of Poetry. What OS are you using and which version ? Also which steps take longest while building ?

[TreyWW]

Hey @Domejko

Thanks for the testing. So I'm on windows 10. From what I can remember the long steps are poetry install and apk installs on Postgres related things. Because these only ever need to be installed once, I believe cache would drastically improve speeds even for me. I'll get some specific build times for each step when I get home

[TreyWW]

Okay so I actually managed to get my build time down from 5 minutes to 39 seconds with multi-stage steps to make use of caches lol and this is still only using 8/16 steps cached.

With caching, it caches literally everything apart from the last step (moving copy . . to right at the end)

I'll provide an updated Dockerfile later, i'm working on a new dockerfile for a new type of deployment at the moment so I dont want to get too mixed up

[Domejko]

Then I think that after updating a Dockerfile if the problem with long build time will persist we can comeback to this topic and search for solution if needed.