Jason Krone

We can cancel this one! I didn't get a resolution, but i think that this is not likely to be a common issue for the community and therefore doesn't warrant...

I didn't wind up solving this but here's a reference that might be helpful https://github.com/foundation-model-stack/foundation-model-stack/blob/4349dacef63e86b6c1acdccb69b48fe562365bb2/fms/models/llama.py#L592

Thanks for adding this (super helpful)! I'm currently running humaneval using your PR. Any ideas why greedy scores change when using different batch sizes? Seems odd to me, and I'm...

@snarayan21 - hoping you may have a suggestion here on how to fix!