Building in Docker fails for alpine/musl

[andyjeffries]

129.3 error: cannot produce cdylib for `commonmarker v1.0.0
129.3 (/usr/local/bundle/gems/commonmarker-1.0.4/ext/commonmarker)` as the target
129.3 `x86_64-unknown-linux-musl` does not support these crate types
129.3 make: *** [Makefile:566: target/release/libcommonmarker.so] Error 101
129.3 
129.3 make failed, exit code 2
129.3 
129.3 Gem files will remain installed in /usr/local/bundle/gems/commonmarker-1.0.4 for
129.3 inspection.
129.3 Results logged to
129.3 /usr/local/bundle/extensions/x86_64-linux-musl/3.3.0/commonmarker-1.0.4/gem_make.out
129.3 
129.3   /usr/local/lib/ruby/3.3.0/rubygems/ext/builder.rb:125:in `run'
129.3   /usr/local/lib/ruby/3.3.0/rubygems/ext/builder.rb:51:in `block in make'
129.3   /usr/local/lib/ruby/3.3.0/rubygems/ext/builder.rb:43:in `each'
129.3   /usr/local/lib/ruby/3.3.0/rubygems/ext/builder.rb:43:in `make'
129.3   /usr/local/lib/ruby/3.3.0/rubygems/ext/ext_conf_builder.rb:42:in `build'
129.3   /usr/local/lib/ruby/3.3.0/rubygems/ext/builder.rb:193:in `build_extension'
129.3 /usr/local/lib/ruby/3.3.0/rubygems/ext/builder.rb:227:in `block in
129.3 build_extensions'
129.3   /usr/local/lib/ruby/3.3.0/rubygems/ext/builder.rb:224:in `each'
129.3   /usr/local/lib/ruby/3.3.0/rubygems/ext/builder.rb:224:in `build_extensions'
129.3   /usr/local/lib/ruby/3.3.0/rubygems/installer.rb:852:in `build_extensions'
129.3 /usr/local/bundle/gems/bundler-2.5.4/lib/bundler/rubygems_gem_installer.rb:76:in
129.3 `build_extensions'
129.3 /usr/local/bundle/gems/bundler-2.5.4/lib/bundler/rubygems_gem_installer.rb:28:in
129.3 `install'
129.3 /usr/local/bundle/gems/bundler-2.5.4/lib/bundler/source/rubygems.rb:205:in
129.3 `install'
129.3 /usr/local/bundle/gems/bundler-2.5.4/lib/bundler/installer/gem_installer.rb:54:in
129.3 `install'
129.3 /usr/local/bundle/gems/bundler-2.5.4/lib/bundler/installer/gem_installer.rb:16:in
129.3 `install_from_spec'
129.3 /usr/local/bundle/gems/bundler-2.5.4/lib/bundler/installer/parallel_installer.rb:132:in
129.3 `do_install'
129.3 /usr/local/bundle/gems/bundler-2.5.4/lib/bundler/installer/parallel_installer.rb:123:in
129.3 `block in worker_pool'
129.3   /usr/local/bundle/gems/bundler-2.5.4/lib/bundler/worker.rb:62:in `apply_func'
129.3 /usr/local/bundle/gems/bundler-2.5.4/lib/bundler/worker.rb:57:in `block in
129.3 process_queue'
129.3   <internal:kernel>:187:in `loop'
129.3 /usr/local/bundle/gems/bundler-2.5.4/lib/bundler/worker.rb:54:in
129.3 `process_queue'
129.3 /usr/local/bundle/gems/bundler-2.5.4/lib/bundler/worker.rb:90:in `block (2
129.3 levels) in create_threads'
129.3 
129.3 An error occurred while installing commonmarker (1.0.4), and Bundler cannot
129.3 continue.
129.3 
129.3 In Gemfile:
129.3   commonmarker
------
ERROR: failed to solve: executor failed running [/bin/sh -c apk add --update --no-cache build-base make gcc musl-dev     && bundle install     && rm -rf /usr/local/bundle/cache/*.gem     && find /usr/local/bundle/gems/ -name "*.c" -delete     && find /usr/local/bundle/gems/ -name "*.o" -delete     && apk del make gcc musl-dev build-base]: exit code: 5

Exited with code exit status 1

This is a Ruby 3.3 (ruby:3.3.0-alpine) based image, that is simply doing a bundle install

[gjtorikian]

Ah yeah I don't support musl, it says so right in the first few lines:

129.3 error: cannot produce cdylib for `commonmarker v1.0.0
129.3 (/usr/local/bundle/gems/commonmarker-1.0.4/ext/commonmarker)` as the target
129.3 `x86_64-unknown-linux-musl` does not support these crate types

I am not sure if I ever will tbh. Bluntly, it's more work to figure out how to get it working in this environment. I have heard too many tales of alpine being "different" enough to not investigate it. I'd consider PRs that support it, though.

[gjtorikian]

Hi, I had a chance to look into this. I can adjust the project to build for x86_64-linux-musl:

cd pkg/commonmarker-1.1.1-x86_64-linux-musl
  Successfully built RubyGem
  Name: commonmarker
  Version: 1.1.1
  File: commonmarker-1.1.1-x86_64-linux-musl.gem

But, I don't know enough about musl to understand what your platform is. The build system I use doesn't recognize it:

❌ Platform x86_64-unknown-linux-musl is not supported

Would support for x86_64-linux-musl be enough for you? What exactly is unknown-linux-musl?

[andyjeffries]

To be honest, we've just pinned the version to 0.23.10 and it works for us. So don't stress too much for now about it (at least from my perspective)

It was being built on an alpine base on CircleCI. So I don't know what CPUs they use or why it comes as "unknown".

[RubenSmit]

We ran into the same problem when building a ruby:3.3.0-alpine based docker container with bundle install. The error it threw was (for reference):

LoadError: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /usr/local/bundle/gems/commonmarker-1.1.5-x86_64-linux/lib/commonmarker/3.3/commonmarker.so) - /usr/local/bundle/gems/commonmarker-1.1.5-x86_64-linux/lib/commonmarker/3.3/commonmarker.so (LoadError)

This is probably caused by trying to run glibc programs in Alpine Linux that uses musl. According to the Alpine linux documentation there are multiple options to do this. We used the gcompat compatibility layer and added this to our Dockerfile:

RUN apk add gcompat

This solved it for us.

[andyjeffries]

We still get an error even after adding gcompat:

#18 109.4 error: cannot produce cdylib for `commonmarker v1.0.0
#18 109.4 (/usr/local/bundle/gems/commonmarker-1.1.5/ext/commonmarker)` as the target
#18 109.4 `aarch64-unknown-linux-musl` does not support these crate types
#18 109.4 make: *** [Makefile:566: target/release/libcommonmarker.so] Error 101
#18 109.4
#18 109.4 make failed, exit code 2

(that was using Docker 4.7.1 on a Mac Sonoma 14.6.1, built using ruby:3.3.0-alpine as a base image)

[RubenSmit]

I can't say what's going wrong without looking at your Dockerfile and docker-compose.yml. We are using ruby:3.3.0-alpine3.19 specifically on docker engine 26.1.4 on several different linux distro's and AWS fargate.

[Ksm125]

Hello, i also have the same error in my docker build using ruby:3.2.2-alpine3.19

 /usr/local/bundle/gems/commonmarker-1.1.5-aarch64-linux/lib/commonmarker/extension.rb:7:in `require_relative': Error relocating /usr/local/bundle/gems/commonmarker-1.1.5-aarch64-linux/lib/commonmarker/3.2/commonmarker.so: __snprintf_chk: symbol not found - /usr/local/bundle/gems/commonmarker-1.1.5-aarch64-linux/lib/commonmarker/3.2/commonmarker.so (LoadError)

here is a sample of my Dockerfile

FROM ruby:3.2.2-alpine3.19

ENV BUNDLER_VERSION=2.4.19

RUN apk upgrade && apk add --no-cache --update build-base \
    curl \
    linux-headers \
    git \
    postgresql-dev \
    postgresql-client \
    vips \
    libheif \
    python3 \
    gcompat

RUN gem install bundler --default -v ${BUNDLER_VERSION}

COPY Gemfile .
COPY Gemfile.lock .

RUN bundle install

[noraj]

I can't get commonmarker 2.0.4 working on docker image ruby:3.4-alpine (I use in my gitlab-ci). I get no error during bundle install but end up with this error when trying to run yard.

$ bundle exec yard doc
[error]: Missing 'commonmarker' gem for Markdown formatting. Install it with `gem install commonmarker`

Also, this is not related to https://github.com/lsegal/yard/issues/1587, https://github.com/lsegal/yard/issues/1528 because I use this in my Gemfile…

  gem 'commonmarker', '~> 2.0' # for markdown support in YARD
  # gem 'yard', ['>= 0.9.27', '< 0.10']
  # https://github.com/lsegal/yard/issues/1528
  gem 'yard', github: 'ParadoxV5/yard', ref: '9e869c940859570b07b81c5eadd6070e76f6291e', branch: 'commonmarker-1.0'

…and I have exactly the same setup on Github Action, the only difference is I used an Ubuntu with glibc image there. While this Alpine use musl, so it's all because commonmarker is not available for that target.

Installing gcompat didn't change anything.

So was someone successfully able to run yard + commonmarker 2.x on a musl based system?

---

$ ruby -v
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [x86_64-linux-musl]

Example of CI: https://gitlab.com/noraj/itdis/-/blob/master/.gitlab-ci.yml

[noraj]

Would support for x86_64-linux-musl be enough for you? What exactly is unknown-linux-musl?

See https://doc.rust-lang.org/rustc/platform-support.html.

x86_64-unknown-linux-musl is 64-bit Linux with musl 1.2.3, while x86_64-linux-musl doesn't seem to exist on Rust Platform Support. x86_64-linux-musl is just https://github.com/oxidize-rb/rb-sys naming. But I think this is the same platform architecture.

[noraj]

For easy cross compilation, can't you use the example provided by rb-sys https://github.com/oxidize-rb/actions/blob/main/cross-gem/readme.md ?

[noraj]

I'd like to politely check if there is any update on the subject as it quite a blocking issue (either change OS and totally rewrite CI for several projects or pin an old version 0.23 and never benefits any commonmaker update).

@gjtorikian Could you give me instructions on how to build commonmarker with x86_64-unknown-linux-musl rust target (x86_64-linux-musl rb-sys target) and adapt Gemfile so I could test inside Alpine musl CI and tell you if it works. I'm willing to help but I'm not familiar with Rust ecosystem.

[gjtorikian]

To test the build out, you can clone the repo and try:

bundle exec rb-sys-dock --platform x86_64-linux-musl

In the past, musl builds with rb-sys were difficult to do, but it looks like recent updates have made this simpler.

I opened a PR to start building for musl: https://github.com/gjtorikian/commonmarker/pull/354

But there are errors, and I have no time right now to work through them.

[gjtorikian]

But there are errors, and I have no time right now to work through them.

Uhhhhh weird! I had errors locally while writing the comment, but it passes in CI. Good enough for me! @noraj if you want to test this out, pin your gem to include-musl. If it works for you, let me know and I'll cut a release.

[noraj]

Oh! Good news.

I may miss something but don't see how pinning the Gemfile to the branch include-musl will change something.

When I check the commits, the gem is unchanged the 3 commits only modify the CI templates and the version of the gem.

https://github.com/gjtorikian/commonmarker/compare/main...include-musl

However, it means the CI may have a build gem with musl support included. On the CI logs (Compile native gem (x86_64-linux-musl, true)) (https://github.com/gjtorikian/commonmarker/actions/runs/13528233054/job/37804034733) I don't see where I can download artifacts. Maybe only you (the owner) can? I see there is rb-sys-dock-v0-x86_64-linux-musl-32cc5904f2b641ed3454d596c17b0b49a2bb61d393e1aae40f3298bb34c0e6f6 in the CI cache https://github.com/gjtorikian/commonmarker/actions/caches, but again I can't download anything.

[gjtorikian]

Oh, you’re so right, I’m sorry. I’m in the middle of my work day and wasn’t thinking right.

I’ll merge the PR and the next version should have a precompiled gem for musl.

[noraj]

Realease job was not triggered https://github.com/gjtorikian/commonmarker/actions/runs/13531446065. Reading the workflow template https://github.com/gjtorikian/commonmarker/blob/main/.github/workflows/tag_and_release.yml I understand it needs a v2.1.0 git tag to trigger the release.

[kivikakk]

I've triggered the jobs on #355, which should hopefully make the release happen! 🤞

[kivikakk]

The release happened, but I'm not sure any musl version got pushed. I think we maybe need to give include_musl: true to https://github.com/gjtorikian/commonmarker/blob/17d01cc47c17f4e784a05c2d3b34339d857acf8d/.github/workflows/tag_and_release.yml#L20 for that to happen.

[gjtorikian]

Yes, sorry! Thank you! v2.1.0 should be on Rubygems for musl now, per 64c43818761b271d97130fe5bc72c70e2ad2f893 and https://github.com/gjtorikian/commonmarker/actions/runs/13535714128

[noraj]

v2.1.0 works on Alpine musl CI

[gjtorikian]

Very glad to hear it!

[noraj]

2.1.1 that twas released for https://github.com/gjtorikian/commonmarker/issues/358, re-introduced the linux-musl issue

[error]: Missing 'commonmarker' gem for Markdown formatting. Install it with gem install commonmarker

Indeed x86_64-linux-musl is not included in 2.1.1, https://rubygems.org/gems/commonmarker/versions:

• 2.1.0 : x86_64-linux-musl & aarch64-linux-musl
• 2.1.1 : aarch64-linux-musl only

But I don't really see what changed: https://github.com/gjtorikian/commonmarker/compare/v2.1.0...v2.1.1

[gjtorikian]

Nothing fundamentally changed in 2.1.1, but the build was not packaging the gem for Ruby 3.3 correctly, so I had to cut a new release to fix that.

Looking at the build logs, I can't help but laugh (from the pain): https://github.com/gjtorikian/commonmarker/actions/runs/13738548919/job/38425110606

Of all the builds, only x86_64-linux-musl randomly and inexplicably failed:

> Publish to RubyGems
Run mkdir -p $HOME/.gem
Pushing gem to https://rubygems.org.../
There was a problem saving your gem. Please try again.
Pushed ./pkg/commonmarker-2.1.1-x86_64-linux-musl.gem

I re-ran the job and confirmed the gem published.