opencompass
opencompass copied to clipboard
[Feature] ensure greedy decoding with huggingface model
Describe the feature
something like
from transformers.generation import GenerationConfig
self.model.generation_config = GenerationConfig.from_pretrained(path, trust_remote_code=True)
self.model.generation_config.do_sample = False
Will you implement it?
- [X] I would like to implement this feature and create a PR!