KenLmStateLayer does not work on sequence (e.g. when optimized)

[albertz]

It's fine if we restrict the usage to only be inside the loop. But in this case, it still could be optimized out, and then it must operate on the sequence.

(Maybe we should also fix this in a more generic way for all layers which do not have this case properly implemented yet?)