Latest lora checkpoints for deepseek-coder/5.7b/mqa-base only generate 1 token to some requests

[hazratisulton]

Use latest lora checkpoint for deepseek-coder/5.7b/mqa-base (iter0010-testloss0.683 in this example) Try first completion. It's working: Accept completion and press Enter:

Expected result: Completion is working and generating code suggestion.

Actual result: Completion not working correctly. Generating only 1 token. In logs messages like this: WEBUI 167.0ms comp-5cc12019689b (arrived to gpu 5.4ms prompt +3.59ms first +5.34ms onebyone +45.81ms/1)

If you are using iter0010-testloss0.683 it is working: Deepseek-coder/1.3b/base loras latest checkpoints finetuned on same code is working.

[yourchanges]

same problem