Automate deployements and updates

[itsvyle]

Hi,

At the moment, the process of updating/reloading the running bots is not automated, and can be quite manual.

Maybe we should consider automating deployements using docker containers. We could use the github packages registry to publish images (storage is unlimited for public repos) and use a service like Watchtower to continuously update the running container as new containers are pushed to the github registry. (there is also a youtube video I watched about self hosting which shows how you can update images in a more direct/raw way, but I can't find it anymore)

The process of pushing new images can be automated via github actions

[aidn3]

This is a good idea. I've never pushed to docker hub before, but we can definitely try making a Dockerfile. I'm not sure how the entire process work. Maybe we need a new workflow to create tags and to push them to docker registry.

[itsvyle]

Hi @aidn3,

I'm glad you like the idea; I believe it would be a good addition to the project as well as an occasion to learn more about continuous deployment.

Your instincts of creating a new workflow for deployments is in my opinion correct, and I have in the past written my own workflow to build and push containers to a registry, as well as create new github tags/releases. You can find the contents of my workflow below; it should be clear enough for you to understand all the steps and provide insight on the best architecture to follow for this repository.

name: Build and deploy
run-name: Building and pushing ${{ inputs.tag_name }}
on:
    workflow_dispatch:
        inputs:
            tag_name:
                description: 'Version name; e.g. 1.0.0; DON''T PUT A V IN FRONT OF IT'
                required: true
            release:
                description: "Whether to create a GitHub Release"
                type: boolean
                default: true
            docker_push:
                description: "Whether to push the Docker image to the registry"
                type: boolean
                default: true
            bypass_cache:
                type: boolean
                default: false
                description: "Disable docker build's use of cache"
            platforms:
                type: string
                default: linux/amd64,linux/arm64
                description: Platforms, comma separated

concurrency:
    group: ${{ github.repository }}
    cancel-in-progress: true

env:
    IMAGE: ghcr.io/itsvyle/dialogv1
    VERSION_TAG: ${{ inputs.tag_name }}

jobs:
    validate_input:
        runs-on: ubuntu-latest
        steps:
            - name: Validate tagname
              id: validate_input
              run: |
                  if [[ ${{ env.VERSION_TAG }} =~ ^v ]]; then
                      echo "Tag name should not start with a v"
                      exit 1
                  fi
                  if ! [[ ${{ env.VERSION_TAG }} =~ ^[0-9]\.[0-9]{1,2}\.[0-9]{1,2}$ ]]; then
                      echo "Version tag does not match the pattern."
                      exit 1
                  fi
            - name: Validate platforms
              run: |
                if ! [[ ${{ inputs.platforms }} =~ ^(linux\/(amd64|arm64))(,*(linux\/(amd64|arm64)))?$ ]]; then
                    echo "Target platform does not match the patterns, it should be one (or more) of 'linux/amd64', 'linux/arm64', comma separated, no spaces"
                fi
    
    build_docker:
        needs: validate_input
        permissions:
            contents: read
            packages: write
        
        runs-on: ubuntu-latest
        if: inputs.docker_push
        strategy:
            fail-fast: true
        steps:
            - name: Checkout
              uses: actions/checkout@v4
            
            - name: "Setup node"
              uses: actions/setup-node@v4
              with:
                cache: yarn
                node-version: '>=20.0.0'
            # Prebuilding/bundling frontend here so that I dont need to do it for each target platform
            - name: "Install dependencies and build frontend"
              run: |
                yarn install
                yarn add sharp -D --ignore-engines
                yarn build
                yarn mail:build
            
            -
             name: Set up Docker Buildx
             uses: docker/setup-buildx-action@v3
             
            # Potentially helps with emultating arm64, I'm not really sure
            -
             name: Set up QEMU
             uses: docker/setup-qemu-action@v3
             with:
                platforms: ${{ inputs.platforms }}
            
            -
             name: Login to Docker registry
             uses: docker/login-action@v3
             with:
                registry: ghcr.io
                username: ${{ github.actor }}
                password: ${{ secrets.GITHUB_TOKEN }}

            # Extracts the metadata from the github repo
            -
             name: Extract metadata
             id: meta
             uses: docker/metadata-action@v5
             with:
                 images: ${{ env.IMAGE }}
                 tags: |
                     type=raw,priority=3,value=${{ env.VERSION_TAG }}
                     type=raw,priority=2,value=latest
             env:
                DOCKER_METADATA_ANNOTATIONS_LEVELS: manifest,index


            - 
             name: Build and push
             uses: docker/build-push-action@v5
             with:
                file: ./Dockerfile.gha
                context: .
                push: true
                no-cache: ${{ inputs.bypass_cache }}
                platforms: ${{ inputs.platforms }}
                tags: ${{ steps.meta.outputs.tags }}
                annotations: ${{ steps.meta.outputs.annotations }}
                cache-from: type=gha
                cache-to: type=gha,mode=max

    create_release:
        runs-on: ubuntu-latest
        if: inputs.release
        needs: build_docker

        permissions:
            contents: write

        steps:
            - name: Create Release
              id: create_release
              uses: softprops/action-gh-release@v2
              env:
                  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
              with:
                  name: v${{ env.VERSION_TAG }} Release
                  tag_name: v${{ env.VERSION_TAG }}
                  draft: true
                  generate_release_notes: true
                  target_commitish: ${{ github.sha }}

A few extra things I'd like to add:

• My example workflow was designed to build a multiplatform container, which may not be necessary here; the cost of building for multiple architectures is that builds can take quite a while (for example, in my repo, the build for linux/amd64 takes 30 seconds, and the build for linux/arm64 takes up to 10 minutes, as arm machine code has to be emulated)
• For bridge, as it's in node.js, the docker images builds should be extremely simple, probably just copying a single bundle file
• Finally, I'd like to touch on your mention of the docker hub; as you may already know, docker hub isn't the only container registry out there, and in my personal project I use github's container registry, as I find it integrates better with github and will associate the container clearly with the repo, at the right of the UI. Do note that should you want to host on docker hub, my example workflow can be extremely easily modified for it (I think simply modifying the image name and login method should do it, although I have not tested this theory)
• Finally, do note that it is possible to automatically trigger a docker push when a new git tag is created

Feel free to reply to this thread or DM me if you have questions!