word-rnn-tensorflow icon indicating copy to clipboard operation
word-rnn-tensorflow copied to clipboard
verified publisher icon hunkim

Score a sentence ?

Open ghaddarAbs opened this issue 9 years ago • 3 comments

Hello, Is there a way to give a score for a gine sentence ??? say a perplexity score

ghaddarAbs avatar Nov 08 '16 19:11 ghaddarAbs

@usr1993 Could you elaborate your point a bit more?

hunkim avatar Nov 09 '16 23:11 hunkim

One way to evaluate a LM is to calculate the perplexity score of a given sentence (how meaningful the sentence is). There is two popular train/dev/test datasets : PTB and One Billion Word Benchmark.

ghaddarAbs avatar Nov 10 '16 10:11 ghaddarAbs

+1

kharazi avatar Nov 16 '16 11:11 kharazi