Repetition Penalty

[joeriakkerman]

Hi there,

I've come to the conclusion that the field repetition_penalty, which can be found here, is not being used. However, this field is supported by the vllm module.

When I checkout the endpoint which is using this request model, I don't see that this field in the model gets mapped within get_gen_params Is there a reason for this? I'd like to test this field out, but it seems not supported and the default value is 1.0.

Our own created chat model is repeating itself over and over again, so I want to checkout if this fields will help us out. When I checkout the vllm code, the docs state the following:

repetition_penalty: Float that penalizes new tokens based on whether they appear in the prompt and the generated text so far. Values > 1 encourage the model to use new tokens, while values < 1 encourage the model to repeat tokens.

So I guess that this field is just what we need, but I can't seem to give this field a different value than 1.0. Is there a reason for this field not being mapped (anymore?)?

Thanks in advance!

[nouf01]

Hi, did you find out how to make the model avoid repetition in it's responses?

[lin-xiaosheng]

I've encountered the same problem as well. It seems that Fastchat might not be correctly receiving the incoming repetition_penalty. Has anyone resolved this issue?

[ssifood]

is this solved?