Incomplete answer to a question

[zxzhijia]

I have a document that has debugging tips for different problems. When I asked the chatbot that queries the document, it give some sentences that is correct in the beginning but it is not able to complete the answer. For example, I have a trouble shoot steps of 5 steps. It just give me the first 3 steps and the last 2 steps are missing. Any parameter that I can tune to fix it?

[pseudotensor]

If I understand your issue, you can change pass --max_new_tokens=512 to generate.py when starting the app. The default of 256 tokens. This also can be controlled from the UI in expert settings. Just be careful, since this eats away at the total context size of 2048 for most models.