bidi-js icon indicating copy to clipboard operation
bidi-js copied to clipboard

Add support for multibyte characters

Open talltyler opened this issue 8 months ago • 3 comments

All functions now operate based on character arrays rather than indexing characters from within the string. The reason why this is important is '👱🏽‍♂️'.length is 7 not 1 so accessing string[0] through string[6] would be modifying the bytes in the middle of this single character which also messed with your logic related to how reordering should work.

This seems to fix https://github.com/lojjic/bidi-js/issues/9 but the problem can be seen more clearly in this modified version of their example where I changed their emoji to the one above. https://codepen.io/talltyler/pen/jOomxwz

To get this working I'm using Intl.Segmenter which is newish but has pretty good support across browsers along with Node 16 and Deno. https://caniuse.com/mdn-javascript_builtins_intl_segmenter If you would rather use something that is more compatible with older environments I can make a change but the only other way I know how to do it is with a huge regex that covers all of the ranges that can connect.

I tried to modify the comments and keep the current api working but if you are ok changing this a few of these functions can be removed.

talltyler avatar Jun 04 '24 13:06 talltyler