about the logprob

Open robinn37 opened this issue 6 years ago • 1 comments

Just a question about the log prob calc in prob interpolation.

https://github.com/smilli/kneser-ney/blob/master/kneser_ney.py#L132

order[kgram] += last_order[suffix] + backoff[prefix]

The original interpolation is based on normal prob instead of log prob. It seems that the above log based interpolation doesn't make sense.

Jun 26 '19 09:06 robinn37

I agree. This seems to be a serious bug.

Oct 09 '21 07:10 cantabile-kwok