Internal Exception escaped out of the interpreter

[Th3-M4jor]

In running some tests with Threads, I managed to trigger this.

Full Stacktrace

(null): markdown.c:2897: void sd_markdown_render(struct buf *, const uint8_t *, size_t, struct sd_markdown *): Assertion `md->work_bufs[BUFFER_BLOCK].size == 0' failed.
java.lang.RuntimeException: Ruby Thread id=102 from ./uncaught_exception.rb:20 terminated with internal error:
        at org.truffleruby.core.thread.ThreadManager.printInternalError(ThreadManager.java:337)
        at org.truffleruby.core.thread.ThreadManager.threadMain(ThreadManager.java:325)
        at org.truffleruby.core.thread.ThreadManager.lambda$initialize$5(ThreadManager.java:293)
        at java.lang.Thread.run(Thread.java:833)
        at com.oracle.truffle.polyglot.PolyglotThread.access$001(PolyglotThread.java:53)
        at com.oracle.truffle.polyglot.PolyglotThread$1.execute(PolyglotThread.java:99)
        at com.oracle.truffle.polyglot.PolyglotThread$ThreadSpawnRootNode.executeImpl(PolyglotThread.java:130)
        at com.oracle.truffle.polyglot.PolyglotThread$ThreadSpawnRootNode.execute(PolyglotThread.java:121)
        at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:656)
        at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:628)
        at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:561)
        at com.oracle.svm.truffle.api.SubstrateOptimizedCallTarget.invokeCallBoundary(SubstrateOptimizedCallTarget.java:115)
        at com.oracle.svm.enterprise.truffle.SubstrateEnterpriseOptimizedCallTarget.a(stripped:284)
        at com.oracle.svm.enterprise.truffle.SubstrateEnterpriseOptimizedCallTarget.doInvoke(stripped:250)
        at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callIndirect(OptimizedCallTarget.java:473)
        at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.call(OptimizedCallTarget.java:454)
        at com.oracle.truffle.polyglot.PolyglotThread.run(PolyglotThread.java:95)
        at com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:705)
        at com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:202)
Caused by: com.oracle.truffle.llvm.runtime.LLVMExitException

truffleruby: an internal exception escaped out of the interpreter,
please report it to https://github.com/oracle/truffleruby/issues.

Ruby Thread id=102 from ./uncaught_exception.rb:20 terminated with internal error: (java.lang.RuntimeException)
        from org.truffleruby.core.thread.ThreadManager.printInternalError(ThreadManager.java:337)
        from org.truffleruby.core.thread.ThreadManager.threadMain(ThreadManager.java:325)
        from org.truffleruby.core.thread.ThreadManager.lambda$initialize$5(ThreadManager.java:293)
        from java.lang.Thread.run(Thread.java:833)
        from com.oracle.truffle.polyglot.PolyglotThread.access$001(PolyglotThread.java:53)
        from com.oracle.truffle.polyglot.PolyglotThread$1.execute(PolyglotThread.java:99)
        from com.oracle.truffle.polyglot.PolyglotThread$ThreadSpawnRootNode.executeImpl(PolyglotThread.java:130)
        from com.oracle.truffle.polyglot.PolyglotThread$ThreadSpawnRootNode.execute(PolyglotThread.java:121)
/usr/local/bin/graalvm-ee-java17-22.2.0/languages/ruby/lib/truffle/truffle/cext.rb:773:in `rb_enc_get'
        from encoding.c:104:in `rb_enc_get'
        from /home/spartan364/.rvm/gems/ext-truffleruby/gems/redcarpet-3.5.1/ext/redcarpet/rc_markdown.c:163:in `rb_redcarpet_md_render'
        from /usr/local/bin/graalvm-ee-java17-22.2.0/languages/ruby/lib/truffle/truffle/cext_ruby.rb:41:in `render'
        from ./uncaught_exception.rb:21:in `<main>'
Caused by:
<no message> (com.oracle.truffle.llvm.runtime.LLVMExitException)
com.oracle.truffle.llvm.libraries.bitcode/src/abort.c:44:in `abort'
        from com.oracle.truffle.llvm.libraries.bitcode/src/assert.c:39:in `__assert_fail'
        from /home/spartan364/.rvm/gems/ext-truffleruby/gems/redcarpet-3.5.1/ext/redcarpet/markdown.c:2897:in `sd_markdown_render'
        from /home/spartan364/.rvm/gems/ext-truffleruby/gems/redcarpet-3.5.1/ext/redcarpet/rc_markdown.c:156:in `rb_redcarpet_md_render'
        from /usr/local/bin/graalvm-ee-java17-22.2.0/languages/ruby/lib/truffle/truffle/cext_ruby.rb:41:in `render'
        from ./uncaught_exception.rb:20:in `block in <main>'

Minimal reproduction:

require 'redcarpet'

str = """
# Hello World

This is a _test_ *markdown* **string**.
"""

class ExtendedRenderer < Redcarpet::Render::HTML
  def paragraph(text)
    "<p>\n#{text}\n</p>\n"
  end
end

renderer = Redcarpet::Markdown.new(ExtendedRenderer)

thr = Thread.new { renderer.render(str) }
renderer.render(str)
thr.join

This occurs with and without the --jvm flag, further a Polyglot::ForeignException will sometimes be thrown instead of there being an uncaught internal exception.

I understand that this error is almost certainly caused by doing something which is not threadsafe.

Strangely though, if I don't extend Redcarpet::Render::HTML and replace one of the methods, no exception is thrown.

[eregon]

Thanks for the report and reproducer. The initial error here seems the Assertion from the first line.

And that's then followed by abort() and that seems to fail because it's one on a non-main Thread. This second part is something we should fix in TruffleRuby, LLVMExitException < AbstractTruffleException so a normal guest (foreign) exception.

Does this happen on CRuby, could you check?

There is a global lock for C extensions by default so that should prevent running any C extension code in parallel.

[Th3-M4jor]

When using ruby 3.0.3p157 (2021-11-24 revision 3fb7d2cadc) [x86_64-linux] no errors occur. Reason likely being as you said, CRuby has a global lock by default for C extensions.

[eregon]

There is a global lock for C extensions by default so that should prevent running any C extension code in parallel.

I meant TruffleRuby has that (by default). And CRuby has global lock for C extension code + Ruby code.

[Th3-M4jor]

Oh, that's what you meant. That probably explains why this error only happens when extending the render class.