Ruby on Rails RedCloth issue when using Docker on Apple M1 hardware

[mo-nathan]

When using the RedCloth gem under Ruby on Rails in a Docker container on an Apple M1 system, the string get truncated just before the first non-ascii character. The problem does not happen on an Apple Intel system or if Rails is run outside of Docker. This GitHub repo should reproduce the issue on any Apple M1 hardware:

https://github.com/mo-nathan/docker-test

I'm guessing it has something to do with how the standard Ruby Docker images are getting created for Apple M1s that triggering some bug in RedCloth. I've tried tracing the issue using a Ruby debugger, but haven't found any additional clues.

I've also posted this issue to Stackoverflow (https://stackoverflow.com/questions/72738254/ruby-on-rails-redcloth-issue-when-using-docker-on-apple-m1-hardware) and to the Ruby Docker GitHub issues page (https://github.com/docker-library/ruby/issues/385).

[markusherzog]

I'm experiencing the same issue:

irb(main):015:0> RedCloth.new("abc def ghi äöü xyz").to_html # or .to_latex
=> "<p>abc def ghi</p>"

Only instead of inside a Docker container, this is in a virtual Debian Parallels box (also on an Apple M1 system)

It works perfectly with the ruby on the Apple host itself, btw.

[lokulin]

@mo-nathan and @markusherzog a work around is to re-compile and install the gem from source.

apt install ragel
gem build redcloth.gemspec
gem install ./RedCloth-4.3.2.gem

Limited testing with some unicode characters seems to indicate that it works. e.g.

#!/usr/bin/env ruby
require 'redcloth'

puts RedCloth::EXTENSION_LANGUAGE

puts RedCloth.new("bbb…ʼɧȪ§©®Ȣƿ").to_html
root@5c4a8cfc51e4:/app# ./tst.rb
C
<p>bbb…ʼɧȪ§©®Ȣƿ</p>

[bvsatyaram]

I see the same issue.

We have recently upgraded our servers. The old servers render the text as expected, but the new servers are trimming the text at the occurrence of the first non-ASCII character.

We used a third-party service for server management. So, we do not exactly know the difference between the packages installed in the older and newer servers.

Both the machines run on Ubuntu 22.04. I tried installing ragel, but it didn't make any difference. Re-compiling the gem from the source is not an option as this would mean that we need to run this on over a dozen servers, and we need to remember to run this every time a new server is spawned.

Script:

require 'redcloth'

puts RedCloth::EXTENSION_LANGUAGE
puts RedCloth.new("Hello 😀, there!").to_html

Output on older machines:

C
<p>Hello</p>

Output on newer machines:

C
<p>Hello 😀, there!</p>

Any help here would be greatly appreciated. Thanks in advance!

[heliocola]

This is really intriguing! Thank y'all for reporting.

I've added all these scenarios here in a test app and they all don't cut. I run on my M2 correctly, and so it runs on CircleCI (in my other app) using Ubuntu 20.04.6 LTS.

I will try to add these scenarios here in the CI for RedCloth and see how it goes. If you have any more details about your environment please send it my way.

[HusseinMorsy]

I have opened a new issue to address a specific problem where RedCloth fails to correctly render non-ASCII characters on arm64 architectures. For more details and to contribute to the discussion, please see Issue #91