Tokenizer
Tokenizer copied to clipboard
Support for citation characters
The tokenizer should support superscripted citation characters. This will also help with GreynirCorrect, which I assume will be heavily used to read student essays and academic papers.