How to continue in multiple generations?

[candymint23]

Hi there, looks like I have an output with no [end of text] for Alpaca and something that has no conclusion, how can we continue it? Does it have a memory? Thanks.

[keldenl]

Nope. You can messing around with the parameters to have it return a longer response. The following has worked well for me (finally after fiddling)

  temp: 0.1,
  top_k: 40,
  top_p: 0.95,
  repeat_last_n: 64,
  repeat_penalty: 1.3,
  n_predict: 256,

You could also try to put it the previous response (prompt + response) and ask it to continue as a workaround, similar to ChatGPT where you just tell it to "continue" when the response is too large