mobicham
mobicham
Btw can you also do a benchmark with speed, based on the logs, it seems this creates an issue with cudagraphs, I am not sure though if this is critical:...
Thanks @lhoestq , related to lm-eval, at least `winogrande`, `mmlu` and `hellaswag`, based on my tests yesterday. But many others like bbh, most probably others too.
Thank you very much @lhoestq , I will try next week 👍