lm-evaluation-harness icon indicating copy to clipboard operation
lm-evaluation-harness copied to clipboard

Removal of trailing newline from description

Open MFajcik opened this issue 7 months ago • 0 comments

Hello, I was debugging where did my trailing whitespace (\n) from the end of description disappeared. It was not present in the input prompt (logged using --log_samples)

And it turns out jinja2 renderer removes it.

If you take a look at Jinja2 environment https://github.com/EleutherAI/lm-evaluation-harness/blob/42dc244867889a19ae80847254a481f446f6e4b7/lm_eval/utils.py#L474

namely, parameter keep_trailing_newline, it is by default set to False (KEEP_TRAILING_NEWLINE = False).

In other words, trailing whitespace disappears here. Is this an expected behavior?

Thank you for providing insights. Best, Martin

MFajcik avatar Jul 29 '24 12:07 MFajcik