Provide musl build

[jminh]

Is it feasible to provide a musl build?

Current atuin-v0.7.1-x86_64-unknown-linux-gnu.tar.gz build cannot run on a CentOS 7 machine due to the glibc issue.

atuin init zsh
atuin: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by atuin)
atuin: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by atuin)

[ellie]

Attempting

cargo build --release --target=x86_64-unknown-linux-musl

On my machine results in an error compiling ring

error: failed to run custom build command for `ring v0.16.20`

Caused by:
  process didn't exit successfully: `/Users/ellie/src/github.com/ellie/atuin/target/release/build/ring-f73322a6feeddaab/build-script-build` (exit status: 101)
  --- stdout
  OPT_LEVEL = Some("3")
  TARGET = Some("x86_64-unknown-linux-musl")
  HOST = Some("aarch64-apple-darwin")
  CC_x86_64-unknown-linux-musl = None
  CC_x86_64_unknown_linux_musl = None
  TARGET_CC = None
  CC = None
  CROSS_COMPILE = None
  CFLAGS_x86_64-unknown-linux-musl = None
  CFLAGS_x86_64_unknown_linux_musl = None
  TARGET_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("false")
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")

  --- stderr
  running "musl-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-U_FORTIFY_SOURCE" "-DNDEBUG" "-c" "-o/Users/ellie/src/github.com/ellie/atuin/target/x86_64-unknown-linux-musl/release/build/ring-dcbe885e6cd9b0ea/out/aesni-x86_64-elf.o" "/Users/ellie/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/pregenerated/aesni-x86_64-elf.S"
  thread 'main' panicked at 'failed to execute ["musl-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-U_FORTIFY_SOURCE" "-DNDEBUG" "-c" "-o/Users/ellie/src/github.com/ellie/atuin/target/x86_64-unknown-linux-musl/release/build/ring-dcbe885e6cd9b0ea/out/aesni-x86_64-elf.o" "/Users/ellie/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/pregenerated/aesni-x86_64-elf.S"]: No such file or directory (os error 2)', /Users/ellie/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/build.rs:653:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Although I am trying to cross-compile for linux + musl on macos, which probably doesn't help 😂 Need to dig into this at some point.

[panekj]

For cross-compilation, you would have to follow rules of each dependency, for ring: https://github.com/briansmith/ring/blob/main/BUILDING.md#cross-compiling

[panekj]

You might be able to do that less painfully with cross (requires docker) https://github.com/ellie/atuin/blob/14b30606a5b2e127f5490f184c8af9ffab780095/.github/workflows/release.yaml#L67

[jirutka]

You might be able to do that less painfully with cross (requires docker)

This is still quite painful. There’s a more straightforward approach, using jirutka/setup-alpine. There’s example for Rust cross-compilation here in the readme and here (and cross-arch here) from one of my projects.

atuin doesn’t link any system libraries by default, so you will not need these hacks to force pkg-config crate do the job right.

[jirutka]

Actually, in this case it’s much easier to use cross (ring is PITA), which you’re already using… it seems to work out of the box, so where’s the problem?

[InCogNiTo124]

I managed to compile atuin for musl just right now:

cargo stdout

$ (~/atuin) cargo build --release --target=x86_64-unknown-linux-musl
   Compiling ring v0.16.20
   Compiling syn v1.0.99
   Compiling parking_lot v0.12.1
   Compiling num-integer v0.1.45
   Compiling getrandom v0.2.7
   Compiling memchr v2.5.0
   Compiling futures-core v0.3.24
   Compiling tracing-core v0.1.30
   Compiling spin v0.5.2
   Compiling generic-array v0.14.6
   Compiling crossbeam-utils v0.8.11
   Compiling ahash v0.7.6
   Compiling log v0.4.17
   Compiling crypto-common v0.1.6
   Compiling block-buffer v0.10.3
   Compiling crossbeam-queue v0.3.6
   Compiling untrusted v0.7.1
   Compiling subtle v2.4.1
   Compiling serde_json v1.0.86
   Compiling tinyvec_macros v0.1.0
   Compiling tinyvec v1.6.0
   Compiling indexmap v1.9.1
   Compiling fnv v1.0.7
   Compiling digest v0.10.5
   Compiling hashbrown v0.12.3
   Compiling vcpkg v0.2.15
   Compiling pin-project-lite v0.2.9
   Compiling http v0.2.8
   Compiling time v0.1.44
   Compiling lock_api v0.4.8
   Compiling tokio v1.25.0
   Compiling futures-channel v0.3.24
   Compiling iana-time-zone v0.1.48
   Compiling scopeguard v1.1.0
   Compiling mio v0.8.4
   Compiling thiserror v1.0.38
   Compiling num_cpus v1.13.1
   Compiling socket2 v0.4.7
   Compiling num-traits v0.2.15
   Compiling regex-syntax v0.6.27
   Compiling unicode-normalization v0.1.21
   Compiling percent-encoding v2.2.0
   Compiling rustversion v1.0.11
   Compiling bytes v1.2.1
   Compiling tower-service v0.3.2
   Compiling futures-sink v0.3.24
   Compiling http-body v0.4.5
   Compiling webpki v0.22.0
   Compiling sct v0.7.0
   Compiling form_urlencoded v1.1.0
   Compiling libsqlite3-sys v0.24.2
   Compiling rustls v0.20.6
   Compiling parking_lot_core v0.8.5
   Compiling dirs-sys v0.3.7
   Compiling slab v0.4.7
   Compiling futures-task v0.3.24
   Compiling instant v0.1.12
   Compiling memoffset v0.7.1
   Compiling unicode-segmentation v1.9.0
   Compiling unicode-bidi v0.3.8
   Compiling httparse v1.8.0
   Compiling smallvec v1.9.0
   Compiling either v1.8.0
   Compiling lazy_static v1.4.0
   Compiling paste v1.0.9
   Compiling ryu v1.0.11
   Compiling futures-util v0.3.24
   Compiling heck v0.4.0
   Compiling rand_core v0.6.3
   Compiling aho-corasick v0.7.19
   Compiling proc-macro-error-attr v1.0.4
   Compiling serde v1.0.145
   Compiling crossbeam-epoch v0.9.13
   Compiling same-file v1.0.6
   Compiling ppv-lite86 v0.2.16
   Compiling ident_case v1.0.1
   Compiling minimal-lexical v0.2.1
   Compiling base64 v0.13.1
   Compiling try-lock v0.2.3
   Compiling cpufeatures v0.2.5
   Compiling strsim v0.10.0
   Compiling async-trait v0.1.58
   Compiling pin-utils v0.1.0
   Compiling nom v7.1.1
   Compiling rand_chacha v0.3.1
   Compiling regex v1.6.0
   Compiling itertools v0.10.5
   Compiling rustls-pemfile v1.0.1
   Compiling want v0.3.0
   Compiling darling_core v0.14.2
   Compiling walkdir v2.3.2
   Compiling dirs v4.0.0
   Compiling parking_lot v0.11.2
   Compiling idna v0.3.0
   Compiling hmac v0.12.1
   Compiling spin v0.9.4
   Compiling proc-macro-error v1.0.4
   Compiling num-bigint v0.2.6
   Compiling unicode_categories v0.1.1
   Compiling byteorder v1.4.3
   Compiling itoa v1.0.3
   Compiling crc-catalog v2.1.0
   Compiling httpdate v1.0.2
   Compiling crc v3.0.0
   Compiling sqlformat v0.2.0
   Compiling chrono v0.4.22
   Compiling url v2.3.1
   Compiling tokio-rustls v0.23.4
   Compiling tokio-stream v0.1.9
   Compiling sqlx-rt v0.6.2
   Compiling hkdf v0.12.3
   Compiling futures-intrusive v0.4.0
   Compiling libsodium-sys v0.2.7
   Compiling dotenvy v0.15.3
   Compiling futures-executor v0.3.24
   Compiling rand v0.8.5
   Compiling serde_derive v1.0.145
   Compiling tokio-macros v1.8.0
   Compiling futures-macro v0.3.24
   Compiling pin-project-internal v1.0.12
   Compiling thiserror-impl v1.0.38
   Compiling tracing-attributes v0.1.23
   Compiling darling_macro v0.14.2
   Compiling hashlink v0.8.0
   Compiling sha2 v0.10.6
   Compiling sha1 v0.10.4
   Compiling atoi v1.0.0
   Compiling stringprep v0.1.2
   Compiling webpki-roots v0.22.4
   Compiling md-5 v0.10.4
   Compiling crossbeam-channel v0.5.6
   Compiling num-rational v0.2.4
   Compiling num-complex v0.2.4
   Compiling num-iter v0.1.43
   Compiling hex v0.4.3
   Compiling whoami v1.2.3
   Compiling bitflags v1.3.2
   Compiling encoding_rs v0.8.31
   Compiling pin-project v1.0.12
   Compiling flume v0.10.14
   Compiling tracing v0.1.37
   Compiling io-lifetimes v1.0.3
   Compiling tower-layer v0.3.2
   Compiling event-listener v2.5.3
   Compiling anyhow v1.0.64
   Compiling mime v0.3.16
   Compiling darling v0.14.2
   Compiling crossbeam-deque v0.8.2
   Compiling axum-core v0.3.2
   Compiling dirs-sys-next v0.1.2
   Compiling memoffset v0.6.5
   Compiling unicode-width v0.1.10
   Compiling openssl-probe v0.1.5
   Compiling rayon-core v1.10.2
   Compiling unicode-xid v0.2.4
   Compiling signal-hook v0.3.14
   Compiling termcolor v1.1.3
   Compiling rustix v0.36.5
   Compiling beef v0.5.2
   Compiling eyre v0.6.8
   Compiling os_str_bytes v6.3.0
   Compiling signature v1.6.1
   Compiling ed25519 v1.5.2
   Compiling clap_lex v0.3.0
   Compiling synstructure v0.12.6
   Compiling rustls-native-certs v0.6.2
   Compiling dirs-next v2.0.0
   Compiling clap_derive v4.0.18
   Compiling derive_builder_core v0.11.2
   Compiling tokio-util v0.7.4
   Compiling tower v0.4.13
   Compiling h2 v0.3.14
   Compiling axum v0.6.4
   Compiling uuid v1.2.1
   Compiling atty v0.2.14
   Compiling thread_local v1.1.4
   Compiling http-range-header v0.3.0
   Compiling linux-raw-sys v0.1.4
   Compiling indenter v0.3.3
   Compiling semver v1.0.14
   Compiling pathdiff v0.2.1
   Compiling ipnet v2.5.0
   Compiling tower-http v0.3.4
   Compiling derive_builder_macro v0.11.2
   Compiling num v0.2.1
   Compiling zeroize_derive v1.3.3
   Compiling clap v4.0.18
   Compiling term v0.7.0
   Compiling rmp v0.8.11
   Compiling logos-derive v0.12.1
   Compiling serde_urlencoded v0.7.1
   Compiling toml v0.5.9
   Compiling serde_path_to_error v0.1.9
   Compiling regex-automata v0.1.10
   Compiling hyper v0.14.20
   Compiling sqlx-core v0.6.2
   Compiling config v0.13.2
   Compiling logos v0.12.1
   Compiling atuin-common v13.0.1 (/home/msmetko/atuin/atuin-common)
   Compiling vte_generate_state_changes v0.1.1
   Compiling terminal_size v0.1.17
   Compiling nix v0.24.3
   Compiling time-core v0.1.0
   Compiling overload v0.1.1
   Compiling fs-err v2.9.0
   Compiling sync_wrapper v0.1.1
   Compiling matchit v0.7.0
   Compiling arrayvec v0.7.2
   Compiling utf8parse v0.2.0
   Compiling nu-ansi-term v0.46.0
   Compiling vte v0.11.0
   Compiling time v0.3.17
   Compiling console v0.15.1
   Compiling tuikit v0.5.0
   Compiling interim v0.1.0
   Compiling matchers v0.1.0
   Compiling chronoutil v0.2.3
   Compiling timer v0.2.0
   Compiling zeroize v1.5.7
   Compiling is-terminal v0.4.1
   Compiling rayon v1.6.1
   Compiling rmp-serde v1.1.1
   Compiling serde_regex v1.1.0
   Compiling signal-hook-mio v0.2.3
   Compiling nix v0.25.1
   Compiling derive_builder v0.11.2
   Compiling parse_duration v2.1.1
   Compiling fuzzy-matcher v0.3.7
   Compiling sql-builder v3.1.1
   Compiling crossbeam v0.8.2
   Compiling filedescriptor v0.8.2
   Compiling defer-drop v1.3.0
   Compiling shellexpand v2.1.2
   Compiling sharded-slab v0.1.4
   Compiling directories v4.0.1
   Compiling pbkdf2 v0.11.0
   Compiling hyper-rustls v0.23.0
   Compiling reqwest v0.11.12
   Compiling base64 v0.21.0
   Compiling rustc-hash v1.1.0
   Compiling number_prefix v0.4.0
   Compiling humantime v2.1.0
   Compiling minspan v0.1.1
   Compiling urlencoding v2.1.2
   Compiling base64 v0.20.0
   Compiling indicatif v0.17.1
   Compiling env_logger v0.10.0
   Compiling tracing-subscriber v0.3.16
   Compiling tiny-bip39 v1.0.0
   Compiling crossterm v0.26.0
   Compiling skim v0.10.2
   Compiling clap_complete v4.0.3
   Compiling runtime-format v0.1.2
   Compiling rpassword v7.0.0
   Compiling cassowary v0.3.0
   Compiling sqlx-macros v0.6.2
   Compiling sodiumoxide v0.2.7
   Compiling sqlx v0.6.2
   Compiling atuin-client v13.0.1 (/home/msmetko/atuin/atuin-client)
   Compiling atuin-server v13.0.1 (/home/msmetko/atuin/atuin-server)
   Compiling atuin v13.0.1 (/home/msmetko/atuin)
    Finished release [optimized] target(s) in 45.49s

everything that was needed was rustup target add x86_64-unknown-linux-musl and the musl library which provides musl-gcc. I think there may be no more blockers - can anyone confirm?