tensorflow_fasttext icon indicating copy to clipboard operation
tensorflow_fasttext copied to clipboard

ngrams maybe contains bug

Open qqhard opened this issue 6 years ago • 0 comments

def GenerateNgrams(words, ngrams):
    nglist = []
    for ng in ngrams:
        for word in words:
            nglist.extend([word[n:n+ng] for n in range(len(word)-ng+1)])
    return ngli

maybe it should like following

def GenerateNgrams(words, ngrams):
    nglist = []
    for ng in ngrams:
        nglist.extend(''.join([words[n:n+ng]) for n in range(len(words)-ng+1)])
    return ngli

qqhard avatar Dec 04 '18 13:12 qqhard