Faster eval

[jmercat]

We might not want to merge this because it is hacky and there might be a usage that I don't foresee that could be impacted.

Problem: Somewhere in llm-foundry or composer, the batches are appended with many stop tokens which makes the eval very slow.

Solution: In OpenLMforCausalLM tests for 0s at the end of the input_ids batch and remove them. Add back fake logits at the end....

Consideration: This hack could be implemented in SimpleComposerOpenLMCausalLM instead and would have less risk to have unforeseen impacts. There shouldn't really be a need for this but I could not understand what was happening in Composer or llm-foundry or find where/why the inputs are appended with 0s.