reiverr icon indicating copy to clipboard operation
reiverr copied to clipboard

Arm64 support, DependaBot alerts, Docker Build Cache

Open just5ky opened this issue 1 year ago • 17 comments

Do Not merge this yet. I overlooked that you were building from docker-compose.yml file instead of DOCKERFILE.

Multiple Updates

  • [x] Arm64 support NEEDS TESTING #41
  • [x] Dependabot alerts to keep up with dependencies. Creates pull requests when there is a new update.
    • https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/about-dependabot-version-updates
  • [x] Added Docker build cache to speedup build time
  • [x] Added Workflow Dispatch to manually trigger CI.

BREAKING CHANGE TO CI

  • [x] By Default, CI will tag the image to branch name. i.e, main
  • [x] Changed the secret name to GIT_TOKEN from GITHUB_TOKEN
    • You will need to rename the secret name in the repository secret.

P.S. Reiverr is amazing, still needs a lot of improvements, but I'm sure it will get there.

just5ky avatar Aug 14 '23 12:08 just5ky

Hey, looks good! Let me know when this is ready to merge. Also If possible, squash commits like "test fix 1".

I'm not familiar with all the actions you used so if you could clarify a couple of things before I merge:

  • Not having worked with dependabot.yml file before, do you recon I should add an entry for package.json, too?
  • Are the QEMU & Buildx actions for the multi-platform functionality? What do they do exactly or what guide did you follow regarding the multi-platform setup
  • overmindtech/buildkit-cache-dance seems really small project, why is it necessary and is it maintained? I would rather use something more widely used to avoid running into issues regarding maintainability.
  • Does your setup still publish the latest tag and the version tag? Are actions still ran automatically whenever I create a tag

No rush, implement all your changes before answering

aleksilassila avatar Aug 14 '23 16:08 aleksilassila

I based this commit thinking you were building the container from DOCKERFILE, but after I created the pull request I noticed that you are using docker-compose.yml to build it, by passing some more commands in the compose file. Now i have to redo the CI file. xD

  • I will quash test fix 1
  • Dependabot.yml provides updates to dependencies, you can add npm, yarn, among many others to it.
    • Example: https://github.com/Just5KY/reiverr/pull/1
  • QEMU is just an emulator to run ARM on top of an x86 system.
  • Buildx provides functionality to build multi-arch containers docker buildx
    • https://docs.docker.com/engine/reference/commandline/buildx_build/
  • Both are needed to build multiarch containers.
  • i added overmindtech/buildkit-cache-dance because you are using ghcr.io and not docker hub to store the container.
    • it caches the container layers, so that in future builds, it can just use cached layers instead of re-building it again. (unless there is any changes)
    • I have experienced that building multiarch containers takes a lot of time with many of my previous containers.
    • I can find others build cache methods, or you can add dockerhub, which makes it really easy.
  • I will have to take a look at the tags again. But the workflow will publish the default tag as the branch name. So if the branch name is main then the container tag will be aleksilassila/reiverr:main

just5ky avatar Aug 14 '23 17:08 just5ky

Okay, gotcha

When it comes to the tags, I think it would be best to have the :latest tag as a convention, as well as the release tag (e.g. v0.3.0) name for now. It makes it more intuitive for people to choose between the latest release and a specific one. Lmk if you run into issues regarding that or if you have any suggestions.

aleksilassila avatar Aug 14 '23 17:08 aleksilassila

@aleksilassila can you explain to me why you are building using docker compose instead of docker build or docker buildx build I'm a confused why there are two different sections in Dockerfile and two compose file. image Why is the development section needed for production build?

just5ky avatar Aug 15 '23 11:08 just5ky

@aleksilassila can you explain to me why you are building using docker compose instead of docker build or docker buildx build

The multiple docker-compose files are there to make docker-compose up build the development environment by targetting the development stage of the dockerfile and docker-compose -f docker-compose.yml -f docker-compose.prod.yml up the production environment by targetting the production stage respectively. The different stages in Dockerfile allow for a) having only one dockerfile for multiple environments and b) smaller production imgage sizes since pre-production stage gets discarded except the build/ directory.

Using docker compose build is probably unnecessary, however. I just used it to easily reuse the values in docker-compose files (container_name, image, target build stage etc). I assume you can pass those with the docker build command.

aleksilassila avatar Aug 15 '23 11:08 aleksilassila

It would be better to separate out the dockerfiles for dev and prod Thats what was causing me to end up with development env. In the final container. And its also not possible to use buildx with docker compose for multiarch builds

just5ky avatar Aug 15 '23 12:08 just5ky

It would be better to separate out the dockerfiles for dev and prod Thats what was causing me to end up with development env. In the final container. And its also not possible to use buildx with docker compose for multiarch builds

If you're not using the docker compose, you'll need to specify the build stage with flags, for example docker build --target production. Does that work with buildx? It seems like at least they have also the --target flag.

https://docs.docker.com/build/building/multi-stage/

https://docs.docker.com/engine/reference/commandline/buildx_build/

aleksilassila avatar Aug 15 '23 12:08 aleksilassila

I think it would I'll give it a try later today.

just5ky avatar Aug 15 '23 12:08 just5ky

okay, so I got it to work with buildx and build a multiarch image. https://github.com/just5ky/reiverr/pkgs/container/reiverr but i am not able to tag it with version. Did you have to create a release first manually, i.e, like v0.4.0 for it to take that as tag

just5ky avatar Aug 15 '23 17:08 just5ky

The way it works currently is I'll run npm version minor/major/patch. It will bump package.json version, create a release commit and add the version number as a tag to that commit. Then I'll push the the tag (and the commit). Every tag that is pushed and that starts with "v" triggers the build.yml. The tag name is extracted from ${{ github.ref_name }}.

name: Build & Deploy

on:
  push:
    tags:
      - v*

# ...

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}
  TAG: ${{ github.ref_name }}

aleksilassila avatar Aug 15 '23 17:08 aleksilassila

okay i see, then i think i got everything working

just5ky avatar Aug 15 '23 17:08 just5ky

I will double-check the code and redo the pull request, from the master branch of my fork.

just5ky avatar Aug 15 '23 17:08 just5ky

Whoops, let me know when this is ready for merge

aleksilassila avatar Aug 20 '23 17:08 aleksilassila

Out on vacation, will be back in a few days.

just5ky avatar Aug 20 '23 18:08 just5ky

Are there any updates on this..? Really looking forward to test Reiverr on my ARM server!

rflrkn avatar May 20 '24 09:05 rflrkn

@rflrkn i stopped working on it after Alex posted about project state https://github.com/aleksilassila/reiverr?tab=readme-ov-file#state-of-the-project-%EF%B8%8F #98

just5ky avatar May 21 '24 19:05 just5ky

Ah, didn't see that. Thanks!

rflrkn avatar May 21 '24 19:05 rflrkn