TonyWeimmer40

Experiencing the same exact issue with the debug version CUBLAS. For context the release version gives buggy output. #1735 The buggy Release version with the gibberish output is far faster...

I've noticed the same issue and in my experience -ins sets default n_keep to 2 even if set to --keep -1 or 0. -p does not have this problem.

I've also noticed n_keep acting unexpectedly. With (-p) n_keep defaults to 0 as expected but with (-ins) it defaults to 2 even if you force --keep -1. Can somebody show...

> I'm going to close this, because while I cant explain why the model kept forgetting the prompt, after continuing to dig through the code for the last week I...