joni fix: invalid unicode input results in endless loop #17

This is based on https://github.com/jruby/joni/pull/21 It only adds the minimum modifications required to make the newly added test case succeed.

Jan 10 '25 23:01 sebthom

@lopex Perhaps this is another case where the macros should be used to check result?

Jan 15 '25 23:01 headius

We're still unsure about this issue. It is reproducible with the example from #17 but we're unsure if this is the best fix. It's not possible to reproduce in Ruby because the bad UTF-8 gets rejected before it ever gets into a regex. If we could reproduce with Onigmo, we could file the issue with them (or with ruby-core) to see how they would fix it.

Mar 03 '25 19:03 headius

I experimented with checking for return value of codeToMbc less than zero and that did not fix the issue (still hangs in a loop). Interestingly, the fix in this PR solves the hanging loop, but fails a few other tests locally (unsure if this is related to recent changes or something I did).

Mar 03 '25 20:03 headius