lighteval
lighteval copied to clipboard
[BUG] : Typo in _continuous_greedy_until causes error with stop_sequence field
Describe the bug
There is a bug in the transformer_models.py _continuous_greedy_until function. The code uses batch[0].stop_sequence, but the correct field should be batch[0].stop_sequences (with an "s" at the end). This typo causes an error when running the function, as the expected field is missing or incorrectly referenced.
To Reproduce
Use the _continuous_greedy_until function with a batch that includes stop sequences. The function tries to access batch[0].stop_sequence, but the batch object defines stop_sequences. An error is raised due to the missing or incorrectly named field.
Expected behavior
The function should correctly reference batch[0].stop_sequences to obtain the list of stop words.
Version info
newest commit version