cadmium
cadmium copied to clipboard
cadmiumcr
Natural Language Processing (NLP) library for Crystal
As you can see browsing Cadmium shards source code, several entities (for lack of a better word) are declared in different locations and in different ways. This issue is not...
## Preface Evaluating the accuracy of the output of an NLP component is a science in itself. When a new NLP algorithm, method or tool is published, it is always...
## Preface As discussed in cadmiumcr/cadmium#31 , `Cadmium::Lemmatizer` needs a `Token` object with POS and morphology data to work properly and be fully tested. The aim of this proposal is...
## Preface Cadmium has a stemmer which is used downstream in several other modules. Its usefulness is not to be questioned. However relying only on a stemmer will limit Cadmium...
The line `item = -item if NEGATORS.includes?(prev_token)` was checking through a Hash, thus it wasn't matching the negator tokens, I changed it to a Set instead. Is this the proper...
Heavily based on [NLTK's implementation](http://www.nltk.org/api/nltk.stem.html#module-nltk.stem.wordnet). Should I add some examples to README.md?
POS tagging is the categorizing of words in a sentence based on part of speech relative to the other words in the sentence. This can be done very simply with...
Spell checking can be accomplished a number of different ways, none of them particularly fast unfortunately. The basic spell checker has a dictionary of words; if any token doesn't match...
A [WER](https://en.m.wikipedia.org/wiki/Word_error_rate) evaluator would be a nice addition to Cadmium IMO. I can work on a PR if you're ok with it. More evaluators can be implemented so I'm planning...