preprocessor
preprocessor copied to clipboard
Include ZWNJ in hashtags
In some RTL languages including Persian, ZWNJ character is used a lot and in particular, it is being used in hastags. For example in "#نیمفاصله" which has ZWNJ between نیم and فاصله.
This line of code needs to be updated and include ZWNJ in the hastag recognition pattern .
https://github.com/s/preprocessor/blob/31d685030831a9ec65d5344dc78677fbcc44c722/preprocessor/defines.py#L34