icu4x
icu4x copied to clipboard
Refactor the upcoming buffer in the collator
CollationElements::upcoming currently holds CharacterAndClassAndTrieValue. Logically its last element (when not at the end of the stream) needs to hold CharacterAndTrieValue (which currently doesn't exist in the collator but exists in the normalizer) and the other elements can be CharacterAndClass. (When we have CharacterAndClass but want CharacterAndTrieValue, we can synthesize the trie value from the class.)
Refactor upcoming so that there's a SmallVec of CharacterAndClass and a separate Option<CharacterAndTrieValue> (that logically comes after the contents of the SmallVec).
This would also remove the special case where at the end of the stream we find an already-decomposed non-starter in the last slot of upcoming and just put it back. After the refactoring, the Option would be None in this case instead of the nature of the last slot changing upon the end of the stream.
@hsivonen What would be an appropriate milestone for this issue?
This doesn't need to be scheduled for any particular milestone, but it also doesn't make sense to intentionally defer this. Let's say tentatively 1.1.