tner
tner copied to clipboard
Prevent punctuation to be included in tokens
Hello,
I have notice that if a work ends with a punctuation mark, the punctuation mark is included as as part of the token.
Is there a way to prevent this behaviour?
Best,
Ed