chinese input to lexer?

[nIxedoahz]

How do I support Chinese string matching

[Mightyjo]

If you encode your Chinese characters in UTF-8 most features will just work. Character classes (i.e. [aAcC-_] will not work, though. They will treat each byte of the code point as a separate character instead of treating them as a multibyte sequence.

You can achieve something similar to character classes using the alternation operator (e.g. "#!" | "?!").