lm-evaluation-harness
lm-evaluation-harness copied to clipboard
Removal of trailing newline from description
Hello, I was debugging where did my trailing whitespace (\n) from the end of description disappeared. It was not present in the input prompt (logged using --log_samples)
And it turns out jinja2 renderer removes it.
If you take a look at Jinja2 environment https://github.com/EleutherAI/lm-evaluation-harness/blob/42dc244867889a19ae80847254a481f446f6e4b7/lm_eval/utils.py#L474
namely, parameter keep_trailing_newline
, it is by default set to False (KEEP_TRAILING_NEWLINE = False).
In other words, trailing whitespace disappears here. Is this an expected behavior?
Thank you for providing insights. Best, Martin