normalise
normalise copied to clipboard
EFord36
A module for normalising text.
`UserWarning: Trying to unpickle estimator LabelPropagation from version 0.18 when using version 0.22. This might lead to breaking code or invalid results. Use at your own risk`
- Normalised was successfully installed using pip - normalise function not recognized in the code - I tried "Import normalise" , received the following error: 'sklearn.semi_supervised.label_propagation'. Installing this module :...
` FutureWarning: The sklearn.semi_supervised.label_propagation module is deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.semi_supervised. Anything that...
I tried to use the spaCy tokenizer, nltk `word_tokenizer`, `sacremoses` `MosesTokenizer`, nltk `TreebankWordTokenizer`, and nltk `TweetTokenizer`. For this example, `"inch BBL, unquote, cost $29.95"` they will all output `['inch', 'BBL',...
IndexError Traceback (most recent call last) in () 7 } 8 ----> 9 pprint(normalise(cleaned_corpus)) 7 frames /usr/local/lib/python3.6/dist-packages/normalise/normalisation.py in normalise(text, tokenizer, verbose, variety, user_abbrevs) 155 return insert(tokenizer(text), verbose=verbose, variety=variety, user_abbrevs=user_abbrevs) 156...
Sometimes you want to disable some preprocessors, like an abbreviation expanding. Add functionality to be able to disable some steps. Because not always you know all the abbreviations that would...
Can use spelling variations, lexical items, incidents of dates where ordering unambiguous (e.g. '15/2/96').