RedditScore icon indicating copy to clipboard operation
RedditScore copied to clipboard

[Bug]: Tutorial example isn't working properly

Open OlehOnyshchak opened this issue 5 years ago • 0 comments

Installation

pip install git+https://github.com/crazyfrogspb/RedditScore.git

Steps to reproduce

from redditscore.tokenizer import CrazyTokenizer

tokenizer = CrazyTokenizer(hashtags=False)
text = "Let's #makeamericagreatagain#americafirst"
print(tokenizer.tokenize(text))

Expected output

["let's", "#makeamericagreatagain", "#americafirst"]

Actual output

['let', "'s", '#makeamericagreatagain', '#americafirst']

Environment

Kaggle's Notebook

OlehOnyshchak avatar Dec 16 '19 22:12 OlehOnyshchak