lda-c icon indicating copy to clipboard operation
lda-c copied to clipboard

How to compute perplexity?

Open jagdeeppani opened this issue 6 years ago • 0 comments

Hi, I am trying to estimate the perplexity on a test set (unseen set of documents). After the inference step on the test set, I see the likelihood file has large negative numbers (e.g. -1800). What are these numbers exactly. If these are log likelihood estimates, should we compute the perplexity by just taking the exponent of average of these values.

Looking forward for an answer. Thanks, Jagdeep

jagdeeppani avatar Jan 10 '19 09:01 jagdeeppani