Support additional server architectures beyond x86_64 and amd64

[seia-soto]

Hello? I recently created a simple script that builds multi-platform supported docker image of shadowbox (or outline-server). During building the script, I got some clean ways to build multi-platform shadowbox image even there is directory called third_party which is platform-sensitive.

Using Docker BuildKit's $TARGETPLATFORM

REF: Here is direct reference to the part of my current multiarch builder(not impl.) script which uses $TARGETPLATFORM.

Thanks to Docker BuildKit's TARGETPLATFORM argument in Dockerfile, we can use it in COPY command inside of Dockerfile.

Like following snippet:

ARG TARGETPLATFORM
RUN mkdir -p third_party
COPY third_party/outline-ss-server/${TARGETPLATFORM} third_party/outline-ss-server
COPY third_party/prometheus/${TARGETPLATFORM} third_party/prometheus

Note that ARG has scoping issue, so we need to put it after FROM keyword.

Following to above, we can expect paths like third_party/${TARGETPLATFORM}:

• third_party/linux/arm64
• third_party/linux/amd64
• third_party/linux/arm/v7
• third_party/linux/arm/v6
• ...

Finally, we can change current third_party directory structure into (only considering docker implementation):

- `${TARGETPLATFORM}`
  - outline-ss-server (binary)
  - prometheus (binary)

// respect current directory structures
- outline-ss-server
  - LICENSE
  - METADATA
- prometheus
  - LICENSE
  - METADATA
- shellcheck (keep original as it is not required at build process)

I agree that this implementation is not completed and should not be used as only works for docker.

Using $(uname -m)

EDIT: Now I have almost complete implementation of this (link to #issuecomment-1021256958).

Or if we target both docker way and non-docker way building, I think it's reasonable to change directory structure into:

- $([[ "$(uname -s)" == "Darwin" ]] && echo "macos" || echo "linux")
  - $(uname -m) # For ARM, it should be arm64 if Apple based, otherwise aarch64
    - outline-ss-server
    - prometheus

: '
- linux
  - x86_64
  - aarch64
  - aarch32? (I could not check it)
- macos
  - x86_64
  - arm64
'

• StackOverflow: Answer of Differences between arm64 and aarch64

Also, modify the build script.

We still need additional changes for docker builds.

# Install third_party dependencies
readonly OS="$([[ "$(uname)" == "Darwin" ]] && echo "macos" || echo "linux")"
## Addition: check arch
readonly ARCH="$(uname -m)"
readonly BIN_DIR="${OUT_DIR}/bin"
mkdir -p "${BIN_DIR}"
cp "${ROOT_DIR}/third_party/prometheus/${OS}/${ARCH}/prometheus" "${BIN_DIR}/"
cp "${ROOT_DIR}/third_party/outline-ss-server/${OS}/${ARCH}/outline-ss-server" "${BIN_DIR}/"

I believe we can make even more simple by modifying build script located in /src/shadowbox/server/build.action.sh.

Ref

• My way to build multi-platform outline-server image (using docker buildx): https://github.com/seia-soto/outline-server-multiarch

---

I am not an expert to the development and want to hear your opinion to this proposal! 👏

[seia-soto]

Updated the title of this issue as the ways are not "completed".

[seia-soto]

I've made some changes on my fork to use Buildx and uname -m. You can compare to the master branch from the following URL: https://github.com/Jigsaw-Code/outline-server/compare/master...seia-soto:add-multiarch-support

EDIT: For version including armv6 support, please refer: https://github.com/Jigsaw-Code/outline-server/compare/master...seia-soto:add-multiarch-support-with-armv6

The above satisfies:

• Both shadowbox/server/build and shadowbox/docker/build action will work on all supported platforms (linux/aarch64, linux/x86_64, linux/armv7l, macos/x86_64, and via Rosetta 2 for M1)
• Won't break for common users to build a single-platform Docker image manually with Buildx by auto-detecting system arch
• Has simple third_party directory structure just like current master branch
• Docker Content Trust

Still, I need to figure out how to use Travis CI to test this as I almost couldn't understand the mechanism of the current Outline-Server's TravisCI script.

Maybe setting up Travis CI would be like...

        # https://www.docker.com/blog/multi-arch-build-what-about-travis/
        - |
          ARCH="$(uname -m)"

          [[ "${ARCH}" == "x86_64" ]] && ARCH="amd64"
          [[ "${ARCH}" == "aarch64" ]] && ARCH="arm64"
          [[ "${ARCH}" == "armv7l" ]] && ARCH="arm-v7"

          mkdir -vp ~/.docker/cli-plugins/
          curl --silent -L "https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.$(uname -s)-${ARCH}" > ~/.docker/cli-plugins/docker-buildx
          chmod a+x ~/.docker/cli-plugins/docker-buildx

          docker buildx create --use

[seia-soto]

For reference, the following output shows the result of uname -m command on GitHub Actions. It's better to include a specific armv6 build but I don't know why the output of the command on linux/arm/v6 is armv7l.

EDIT: Note that I added an implementation with armv6 support but this is more dirty solution as there are many if statements in scripts: https://github.com/Jigsaw-Code/outline-server/compare/master...seia-soto:add-multiarch-support-with-armv6 Also, I copy-pasted the function called remap_arch in two files as it's better than beating with path problem in the shell.

#10 [linux/arm64 2/2] RUN uname -a
#10 0.719 Linux buildkitsandbox 5.11.0-1027-azure #30~20.04.1-Ubuntu SMP Wed Jan 12 20:56:50 UTC 2022 aarch64 Linux
#10 DONE 0.7s

#13 [linux/amd64 2/2] RUN uname -a
#13 0.638 Linux buildkitsandbox 5.11.0-1027-azure #30~20.04.1-Ubuntu SMP Wed Jan 12 20:56:50 UTC 2022 x86_64 Linux
#13 DONE 0.7s

#14 [linux/arm/v7 2/2] RUN uname -a
#14 0.707 Linux buildkitsandbox 5.11.0-1027-azure #30~20.04.1-Ubuntu SMP Wed Jan 12 20:56:50 UTC 2022 armv7l Linux
#14 DONE 0.8s

#12 [linux/arm/v6 2/2] RUN uname -a
#12 0.752 Linux buildkitsandbox 5.11.0-1027-azure #30~20.04.1-Ubuntu SMP Wed Jan 12 20:56:50 UTC 2022 armv7l Linux
#12 DONE 0.8s

Files to test

/Dockerfile

FROM alpine:latest

RUN uname -a

/.github/workflows/test.yml

name: Test

on:
  push:

jobs:
  test:
    runs-on: ubuntu-20.04
    steps:
      - name: Checkout repository
        uses: actions/checkout@v2

      - uses: actions/setup-node@v2
        with:
          node-version: ${{ matrix.node }}

      - name: Setup QEMU
        uses: docker/setup-qemu-action@v1
        with:
          image: tonistiigi/binfmt:latest
          platforms: all

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1

      - name: Build
        run: |
          docker buildx build \
            --platform="linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6" \
            --force-rm \
            --file Dockerfile \
            --tag "test" \
            .

However, supporting only armv7l would be enough as uname -m results armv7l on RPi 3 (for Oracle Ampere instances, uname -m shows aarch64 as both OS and CPU are 64 bit). In other words, we can expect most users are ready to use Outline on their ARM computing power.

Refs

• Possible values for $ uname -m: https://stackoverflow.com/questions/45125516/possible-values-for-uname-m
• On a 64-bit processor, you'd see a string starting with armv8 (or above) if the uname process itself is a 32-bit process, or aarch64 if it's a 64-bit process.: https://unix.stackexchange.com/a/136519

[seia-soto]

Congrats! 🥳

Now I have a test suite that verifies the ways above work.

add-multiarch-support

This way uses pure $(uname -m) command to pick up valid third party binaries. Still, I needed to remap the value into Docker-recognizable text like below but it's cleaner than add-multiarch-support-with-armv6:

# Install third_party dependencies
if [[ "$(uname)" == "Darwin" ]]; then
  readonly OS="macos"
  readonly ARCH="x86_64"
else
  readonly OS="linux"
  readonly ARCH="$(uname -m)"
fi

readonly BIN_DIR="${OUT_DIR}/bin"

mkdir -p "${BIN_DIR}"
cp "${ROOT_DIR}/third_party/prometheus/${OS}/prometheus_${ARCH}" "${BIN_DIR}/prometheus"
cp "${ROOT_DIR}/third_party/outline-ss-server/${OS}/outline-ss-server_${ARCH}" "${BIN_DIR}/outline-ss-server"

As above code describes, we just put value from $(uname -m) directly into third_party directory.

Tests

• x86_64: https://github.com/seia-soto/outline-server/runs/4946703360
• aarch64: https://github.com/seia-soto/outline-server/runs/4946703387
• armv7l: https://github.com/seia-soto/outline-server/runs/4946703426

add-multiarch-support-with-armv6 (dirtier but more coverage)

Like above add-multiarch-support but adds armv6 compatibility by adding some customizable parameters on build scripts. For brief diff, I added remap_arch functions and allowed customized value:

# Detect and set architecture for general users to build without installing emulator.
remap_arch() {
  local ARCH="${1}" AMD64="${2:-amd64}" ARM64="${3:-arm64}" ARMv7="${4:-armv7}" ARMv6="${5:-armv6}"

  [[ "${ARCH}" == *"amd64"* || "${ARCH}" == *"x86_64"* ]] && ARCH="${AMD64}"
  [[ "${ARCH}" == *"arm64"* || "${ARCH}" == *"aarch64"* ]] && ARCH="${ARM64}"
  [[ "${ARCH}" == *"v7"* ]] && ARCH="${ARMv7}"
  [[ "${ARCH}" == *"v6"* ]] && ARCH="${ARMv6}"

  echo "${ARCH}"
}

# Specify the target platform with `$SB_PLATFORM`.
[[ -z "${SB_PLATFORM:-}" ]] && SB_PLATFORM="$(remap_arch "$(uname -m)" linux/amd64 linux/arm64 linux/arm/v7 linux/arm/v6)"

I know that's a bit dirty. However, in this way, we can extend the coverage even there are many variants in $(uname -m) output.

Tests

• x86_64: https://github.com/seia-soto/outline-server/runs/4946927874
• aarch64: https://github.com/seia-soto/outline-server/runs/4946927902
• armv7: https://github.com/seia-soto/outline-server/runs/4946927923
• armv6: https://github.com/seia-soto/outline-server/runs/4946927942

[Blackhatfrance]

Please update the dockerimage

[seia-soto]

Please update the dockerimage

Hi, @Blackhatfrance . I'm here to tell you the issue is fixed and now I am deploying with three tags. However, this place is for Outline-Server, so I recommend you to continue on issue in my repo. :)

[Blackhatfrance]

Thanks @seia-soto

[ghost]

+1 need support ARM 64 server

[maddyhof]

@seia-soto, are you interested in submitting this code to the Outline repo? If so, please open a pull request. Thanks for considering it!