`tz_world.update` taking over double the memory on OTP 27.0.1

[Jdyn]

EDIT: I narrowed it down further and was able to build on 1.17.2 and OTP 26.2.5.2. So it looks like the problem is OTP 27.0.1

I am on tz_world 1.3.3

Hey, i've updated to 1.17 but am unable to deploy due to a significant increase in memory usage when running tz_world.update compared to 1.15.

Here are the two docker images. 1.15.7 builds perfectly and the image with 1.17 OOMs after 4gb of usage.

Elixir 1.15.7 OTP 25.3.2.7 Working image

  # Find eligible builder and runner images on Docker Hub. We use Ubuntu/Debian
  # instead of Alpine to avoid DNS resolution issues in production.
  #
  # https://hub.docker.com/r/hexpm/elixir/tags?page=1&name=ubuntu
  # https://hub.docker.com/_/ubuntu?tab=tags
  #
  # This file is based on these images:
  #
  #   - https://hub.docker.com/r/hexpm/elixir/tags - for the build image
  #   - https://hub.docker.com/_/debian?tab=tags&page=1&name=bullseye-20231009-slim - for the release image
  #   - https://pkgs.org/ - resource for finding needed packages
  #   - Ex: hexpm/elixir:1.15.7-erlang-25.3.2.7-debian-bullseye-20231009-slim
  #
  ARG ELIXIR_VERSION=1.15.7
  ARG OTP_VERSION=25.3.2.7
  ARG DEBIAN_VERSION=bullseye-20231009-slim

  ARG BUILDER_IMAGE="hexpm/elixir:${ELIXIR_VERSION}-erlang-${OTP_VERSION}-debian-${DEBIAN_VERSION}"
  ARG RUNNER_IMAGE="debian:${DEBIAN_VERSION}"

  FROM ${BUILDER_IMAGE} as builder

  # install build dependencies
  RUN apt-get update -y && apt-get install -y build-essential git \
      && apt-get clean && rm -f /var/lib/apt/lists/*_*

  # prepare build dir
  WORKDIR /app

  # install hex + rebar
  RUN mix local.hex --force && \
      mix local.rebar --force

  # set build ENV
  ENV MIX_ENV="prod"

  # install mix dependencies
  COPY mix.exs mix.lock ./
  RUN mix deps.get --only $MIX_ENV
  RUN mkdir config

  # copy compile-time config files before we compile dependencies
  # to ensure any relevant config change will trigger the dependencies
  # to be re-compiled.
  COPY config/config.exs config/${MIX_ENV}.exs config/
  RUN mix deps.compile

  COPY priv priv

  COPY lib lib

  # Compile the release
  RUN mix compile

  # Changes to config/runtime.exs don't require recompiling the code
  COPY config/runtime.exs config/

  COPY rel rel
  RUN mix tz_world.update
  RUN mix release

  # start a new build stage so that the final image will only contain
  # the compiled release and other runtime necessities
  FROM ${RUNNER_IMAGE}

  RUN apt-get update -y && \
    apt-get install -y libstdc++6 openssl libncurses5 locales ca-certificates \
    && apt-get clean && rm -f /var/lib/apt/lists/*_*

  # Set the locale
  RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen

  ENV LANG en_US.UTF-8
  ENV LANGUAGE en_US:en
  ENV LC_ALL en_US.UTF-8

  WORKDIR "/app"
  RUN chown nobody /app

  # set runner ENV
  ENV MIX_ENV="prod"

  # Only copy the final release from the build stage
  COPY --from=builder --chown=nobody:root /app/_build/${MIX_ENV}/rel/nimble ./

  USER nobody

  # If using an environment that doesn't automatically reap zombie processes, it is
  # advised to add an init process such as tini via `apt-get install`
  # above and adding an entrypoint. See https://github.com/krallin/tini for details
  # ENTRYPOINT ["/tini", "--"]

  CMD ["/app/bin/server"]

Elixir 1.17.2 OTP 27.0.1 OOM

  # Find eligible builder and runner images on Docker Hub. We use Ubuntu/Debian
  # instead of Alpine to avoid DNS resolution issues in production.
  #
  # https://hub.docker.com/r/hexpm/elixir/tags?page=1&name=ubuntu
  # https://hub.docker.com/_/ubuntu?tab=tags
  #
  # This file is based on these images:
  #
  #   - https://hub.docker.com/r/hexpm/elixir/tags - for the build image
  #   - https://hub.docker.com/_/debian?tab=tags&page=1&name=bullseye-20231009-slim - for the release image
  #   - https://pkgs.org/ - resource for finding needed packages
  #   - Ex: hexpm/elixir:1.15.7-erlang-25.3.2.7-debian-bullseye-20231009-slim
  #
  ARG ELIXIR_VERSION=1.17.2
  ARG OTP_VERSION=27.0.1
  ARG DEBIAN_VERSION=buster-20240612-slim

  ARG BUILDER_IMAGE="hexpm/elixir:${ELIXIR_VERSION}-erlang-${OTP_VERSION}-debian-${DEBIAN_VERSION}"
  ARG RUNNER_IMAGE="debian:${DEBIAN_VERSION}"

  FROM ${BUILDER_IMAGE} as builder

  # install build dependencies
  RUN apt-get update -y && apt-get install -y build-essential git \
      && apt-get clean && rm -f /var/lib/apt/lists/*_*

  # prepare build dir
  WORKDIR /app

  # install hex + rebar
  RUN mix local.hex --force && \
      mix local.rebar --force

  # set build ENV
  ENV MIX_ENV="prod"

  # install mix dependencies
  COPY mix.exs mix.lock ./
  RUN mix deps.get --only $MIX_ENV
  RUN mkdir config

  # copy compile-time config files before we compile dependencies
  # to ensure any relevant config change will trigger the dependencies
  # to be re-compiled.
  COPY config/config.exs config/${MIX_ENV}.exs config/
  RUN mix deps.compile

  COPY priv priv

  COPY lib lib

  # Compile the release
  RUN mix compile

  # Changes to config/runtime.exs don't require recompiling the code
  COPY config/runtime.exs config/

  COPY rel rel
  RUN mix tz_world.update
  RUN mix release

  # start a new build stage so that the final image will only contain
  # the compiled release and other runtime necessities
  FROM ${RUNNER_IMAGE}

  RUN apt-get update -y && \
    apt-get install -y libstdc++6 openssl libncurses5 locales ca-certificates \
    && apt-get clean && rm -f /var/lib/apt/lists/*_*

  # Set the locale
  RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen

  ENV LANG en_US.UTF-8
  ENV LANGUAGE en_US:en
  ENV LC_ALL en_US.UTF-8

  WORKDIR "/app"
  RUN chown nobody /app

  # set runner ENV
  ENV MIX_ENV="prod"

  # Only copy the final release from the build stage
  COPY --from=builder --chown=nobody:root /app/_build/${MIX_ENV}/rel/nimble ./

  USER nobody

  # If using an environment that doesn't automatically reap zombie processes, it is
  # advised to add an init process such as tini via `apt-get install`
  # above and adding an entrypoint. See https://github.com/krallin/tini for details
  # ENTRYPOINT ["/tini", "--"]

  CMD ["/app/bin/server"]

• The first spike is the attempt at building 1.17.2. Note that the 1.17 build crashes and doesn't complete because I have a 4gb build limit, so it is likely using even more memory.
• The second spike later on is the build on 15.7 which completes entirely.

Any ideas?