sonic
sonic copied to clipboard
valeriansaliou
[Docker] Build image for arm64v8
I think it would be cool if we could use Sonic on raspberry pi (I'm thinking of raspberry pi 3 and up that runs in 64 bits)
Thanks,
I've not managed to build a MUSL binary for any ARM platform, due to dependencies incompatibility. If someone manages to build it for an ARM target, I'll accept a PR.
This issue is 2 years old. I'm sure by now musl is buildable on ARM64v8? I#d love to get this running as docker image on my RaspberryPi for ArchiveBox
I've tried 3 months ago w/ the latest RocksDB and could not successfully have a MUSL build working. I managed to compile it through some tricks, but Sonic segfaulted at runtime, and unfortunately I was out of time and needed to move to other things so I didn't keep on looking for a fix since then. I'll happily take PRs on that one!
It can be built now by replacing "x86_64" to "aarch64" in the Dockerfile.
FROM rust:slim-bullseye AS build
RUN apt-get update
RUN apt-get install -y build-essential clang
RUN rustup --version
RUN rustup component add rustfmt
RUN rustc --version && \
rustup --version && \
cargo --version
WORKDIR /app
COPY . /app
RUN cargo clean && cargo build --release --target aarch64-unknown-linux-gnu
RUN strip ./target/aarch64-unknown-linux-gnu/release/sonic
FROM debian:bullseye-slim
WORKDIR /usr/src/sonic
COPY --from=build /app/target/aarch64-unknown-linux-gnu/release/sonic /usr/local/bin/sonic
CMD [ "sonic", "-c", "/etc/sonic.cfg" ]
EXPOSE 1491
This is for Apple Silicon CPUs if I understand well the purpose of aarch64, would it also work on ARMv8 CPUs?
AArch64 is the 64-bit state introduced in the ARMv8-A architecture. ARM ARMv8 CPU architecture has two ISAs (instruction sets). AArch64 and AArch32 are the two instruction sets that ARMv8-A can support. According to Wikipedia we can say they are the same.
AArch64 or ARM64 is the 64-bit extension of the ARM architecture.
I built the Dockerfile above with my Raspberry Pi 4 64-bit . So yeah it works on ARMv8 as far as i can see.
Would love to see this get merged soon! Also wanting to index my Archivebox data on an RPi. 🖤
It can be built now by replacing "x86_64" to "aarch64" in the Dockerfile.
FROM rust:slim-bullseye AS build RUN apt-get update RUN apt-get install -y build-essential clang RUN rustup --version RUN rustup component add rustfmt RUN rustc --version && \ rustup --version && \ cargo --version WORKDIR /app COPY . /app RUN cargo clean && cargo build --release --target aarch64-unknown-linux-gnu RUN strip ./target/aarch64-unknown-linux-gnu/release/sonic FROM debian:bullseye-slim WORKDIR /usr/src/sonic COPY --from=build /app/target/aarch64-unknown-linux-gnu/release/sonic /usr/local/bin/sonic CMD [ "sonic", "-c", "/etc/sonic.cfg" ] EXPOSE 1491
I just tries your code in a SOC arm64 bits and get this result:
#0 6.055 Updating crates.io index
#0 523.3 Downloading crates ...
#0 524.1 Downloaded fst-regex v0.3.0
#0 524.1 Downloaded fxhash v0.2.1
#0 524.1 Downloaded phf_shared v0.11.1
#0 524.2 Downloaded lazycell v1.3.0
#0 524.5 Downloaded rand_chacha v0.3.1
#0 524.8 Downloaded strsim v0.10.0
#0 524.8 Downloaded siphasher v0.3.10
#0 524.8 Downloaded linked_hash_set v0.1.4
#0 524.8 Downloaded lazy_static v1.4.0
#0 524.9 Downloaded version_check v0.9.4
#0 525.1 Downloaded shlex v1.1.0
#0 525.2 Downloaded rustc-hash v1.1.0
#0 525.5 Downloaded phf_generator v0.11.1
#0 525.7 Downloaded clap_lex v0.2.4
#0 526.0 Downloaded peeking_take_while v0.1.2
#0 526.0 Downloaded fst-levenshtein v0.3.0
#0 526.1 Downloaded phf_codegen v0.11.1
#0 526.2 Downloaded glob v0.3.0
#0 526.3 Downloaded utf8-ranges v1.0.5
#0 526.4 Downloaded cfg-if v1.0.0
#0 526.4 Downloaded atty v0.2.14
#0 526.4 Downloaded cfg-if v0.1.10
#0 526.5 Downloaded radix v0.6.0
#0 526.6 Downloaded jobserver v0.1.25
#0 526.6 Downloaded rand_core v0.6.4
#0 526.7 Downloaded once_cell v1.15.0
#0 526.7 Downloaded ppv-lite86 v0.2.16
#0 526.8 Downloaded twox-hash v1.6.3
#0 526.9 Downloaded tikv-jemallocator v0.4.3
#0 526.9 Downloaded cedarwood v0.4.6
#0 527.0 Downloaded smallvec v1.10.0
#0 527.0 Downloaded termcolor v1.1.3
#0 527.1 Downloaded autocfg v1.1.0
#0 527.1 Downloaded byteorder v1.4.3
#0 527.1 Downloaded getrandom v0.2.7
#0 527.2 Downloaded linked-hash-map v0.5.6
#0 527.5 Downloaded phf v0.11.1
#0 527.8 Downloaded cexpr v0.6.0
#0 528.0 Downloaded fs_extra v1.2.0
#0 528.0 Downloaded bitflags v1.3.2
#0 528.1 Downloaded memmap v0.6.2
#0 528.2 Downloaded log v0.4.17
#0 528.4 Downloaded libloading v0.7.3
#0 528.5 Downloaded ahash v0.7.6
#0 528.5 Downloaded unicode-ident v1.0.5
#0 528.5 Downloaded indexmap v1.9.1
#0 528.6 Downloaded clang-sys v1.4.0
#0 528.6 Downloaded static_assertions v1.1.0
#0 528.8 Downloaded os_str_bytes v6.3.0
#0 529.2 Downloaded quote v1.0.21
#0 529.7 Downloaded proc-macro2 v1.0.47
#0 529.8 Downloaded serde_derive v1.0.145
#0 530.3 Downloaded textwrap v0.15.1
#0 530.6 Downloaded hashbrown v0.12.3
#0 530.7 Downloaded memchr v2.5.0
#0 530.8 Downloaded toml v0.5.9
#0 530.9 Downloaded cc v1.0.73
#0 531.1 Downloaded serde v1.0.145
#0 531.6 Downloaded rand v0.8.5
#0 531.7 Downloaded rocksdb v0.17.0
#0 532.1 Downloaded minimal-lexical v0.2.1
#0 532.2 Downloaded whatlang v0.16.1
#0 532.2 Downloaded unicode-segmentation v1.10.0
#0 532.5 Downloaded nom v7.1.1
#0 532.5 Downloaded aho-corasick v0.7.19
#0 533.0 Downloaded regex-syntax v0.3.9
#0 534.1 Downloaded bindgen v0.59.2
#0 535.3 Downloaded nix v0.18.0
#0 535.5 Downloaded clap v3.2.22
#0 537.4 Downloaded syn v1.0.102
#0 538.0 Downloaded regex-syntax v0.6.27
#0 538.1 Downloaded regex v1.6.0
#0 552.4 Downloaded libc v0.2.135
#0 553.6 Downloaded tikv-jemalloc-sys v0.4.3+5.2.1-patched.2
#0 587.6 Downloaded fst v0.3.5
#0 612.0 Downloaded jieba-rs v0.6.7
#0 617.0 Downloaded librocksdb-sys v6.20.3
#0 622.1 Compiling libc v0.2.135
#0 622.1 Compiling cfg-if v1.0.0
#0 622.1 Compiling memchr v2.5.0
#0 628.7 Compiling proc-macro2 v1.0.47
#0 629.1 Compiling unicode-ident v1.0.5
#0 631.9 Compiling glob v0.3.0
#0 633.6 error: could not compile `proc-macro2`
#0 633.6
#0 633.6 Caused by:
#0 633.6 process didn't exit successfully: `rustc --crate-name build_script_build --edition=2018 /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro2-1.0.47/build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg 'feature="default"' --cfg 'feature="proc-macro"' -C metadata=d726feb61c0c13f7 -C extra-filename=-d726feb61c0c13f7 --out-dir /app/target/release/build/proc-macro2-d726feb61c0c13f7 -C strip=symbols -L dependency=/app/target/release/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)
#0 633.6 warning: build failed, waiting for other jobs to finish...
------
Dockerfile:15
--------------------
13 | WORKDIR /app
14 | COPY . /app
15 | >>> RUN cargo clean && cargo build --release --target aarch64-unknown-linux-gnu
16 | RUN strip ./target/aarch64-unknown-linux-gnu/release/sonic
17 |
--------------------
ERROR: failed to solve: process "/bin/sh -c cargo clean && cargo build --release --target aarch64-unknown-linux-gnu" did not complete successfully: exit code: 101
any news on this? I was also able to build the image. Would be nice to have an official one :blush:
I'll batch-process all my OSS projects MR soon, pbb in a few weeks. Sorry I didn't have any chance to review this before.
It can be built now by replacing "x86_64" to "aarch64" in the Dockerfile.
FROM rust:slim-bullseye AS build RUN apt-get update RUN apt-get install -y build-essential clang RUN rustup --version RUN rustup component add rustfmt RUN rustc --version && \ rustup --version && \ cargo --version WORKDIR /app COPY . /app RUN cargo clean && cargo build --release --target aarch64-unknown-linux-gnu RUN strip ./target/aarch64-unknown-linux-gnu/release/sonic FROM debian:bullseye-slim WORKDIR /usr/src/sonic COPY --from=build /app/target/aarch64-unknown-linux-gnu/release/sonic /usr/local/bin/sonic CMD [ "sonic", "-c", "/etc/sonic.cfg" ] EXPOSE 1491
This works for me when running on github actions.
Also is there an update on this? Adding arm is working for me.
