icu4x icon indicating copy to clipboard operation
icu4x copied to clipboard

Provide a trie-based alternative to UnicodeSet

Open hsivonen opened this issue 1 year ago • 1 comments

The ICU4X composing normalizer uses a UnicodeSet for a fast-path pass-through check while the ICU4C composing normalizer uses a code point trie lookup. ICU4C ends up being faster ever after optimizing other aspects on the ICU4X side, including special-casing the lowest range of the set (the Latin range below the combining diacritics block).

For a known-fragmented compile-time-known set, we should provide an alternative to UnicodeSet that uses the structure of CodePointTrie, but instead of wasting 7 bits of each value byte, divides the length of the value array by 8 and stores 8 logical bits in each byte.

hsivonen avatar Jul 20 '22 13:07 hsivonen

For the normalizer, #2221 makes more sense.

hsivonen avatar Jul 20 '22 16:07 hsivonen