kenlm icon indicating copy to clipboard operation
kenlm copied to clipboard

Detailed Usage of lm.State

Open xiabingquan opened this issue 3 years ago • 0 comments

in python/example.py

#Stateful query
state = kenlm.State()
state2 = kenlm.State()
#Use <s> as context.  If you don't want <s>, use model.NullContextWrite(state).
model.BeginSentenceWrite(state)
accum = 0.0
accum += model.BaseScore(state, "a", state2)
accum += model.BaseScore(state2, "sentence", state)

I tried using the same State variable along the whole process. That's to say:

accum = 0.0
accum += model.BaseScore(state, "a", state)
accum += model.BaseScore(state, "sentence", state)

It turns out that this alternative doesn't work. I'm not quite familiar with C++ so I didn't read the source code. Could anybody provide some hints or link to related docs on how we should use State? Any help is appreciated.Thanks!

xiabingquan avatar Jul 06 '22 14:07 xiabingquan