unicode-transforms
unicode-transforms copied to clipboard
composewell
Fast Unicode normalization in Haskell
See GHC ticket https://gitlab.haskell.org/ghc/ghc/-/issues/14072 . We should also mention the GHC version on which the benchmarks in the README are measured.
Implement [Quick Check algorithm](https://www.unicode.org/reports/tr15/#Detecting_Normalization_Forms). __This is very much WIP__; it is intended to open the discussion on the implementation. It relies on an [ongoing PR of `unicode-data`](https://github.com/composewell/unicode-data/pull/72). - [ ]...
DRY them. Use common stanzas and cleanup the file in general.
According to its documentation, text-icu's collation algorithm uses incremental normalization. This is very helpful in collation: when you're comparing two strings, the decision about how to order them is generally...
The test comparing with icu, closed by #46 has failed again on master branch CI: ``` Failures: test/Properties.hs:64:9: 1) Comparing random strings with ICU... Falsified (after 2753 tests and 6...
Currently we need to do three lookups: * is it decomposable? * if not decomposable: * is it combining? * combining class when reordering We can have a single lookup...
Need to implement unicode conformance testing. See https://www.unicode.org/reports/tr15/tr15-18.html#Conformance%20Testing .
Decompose code is well optimized but compose still has a lot of scope for optimization. Though its performance is close to `utf8proc` that we were using earlier, it is still...