unicode support

Open CrockAgile opened this issue 11 months ago • 2 comments

A contributor was unclear while working on #154 if the crate supported unicode (link).

Documentation and tests should call this out that BNF intends (and should?) support unicode.

Jan 05 '25 00:01 CrockAgile

i think that we should definitely support grammars that operate on unicode, but for the grammar definitions themselves limiting unicode support to terminals could be an avenue for performance gains + make the code a bit simpler (both in human and machine terms)

Jan 05 '25 02:01 Carlyle-Foster

it's not a pressing issue though, rn i think we handle unicode correctly and use it everywhere

Jan 05 '25 02:01 Carlyle-Foster