ChrisSpraaklab

@gante Thanks for your quick response. However, what I mean is that when input_ids_seq_length is set to input_ids.shape[-1], this value is always equal to 1 (as it comes from _prepare_decoder_input_ids_for_generation)....

Thanks! Your solution does indeed produce the result I was looking for. I was just quite confused about the naming convention and documentation around max_new_tokens. I was under the impression...