Srikanth Srungarapu

I'll try debugging this on my end and post solution here if I solve the issue. Thanks for quick reply!

Currently, benchllama supports evaluating FIM models. Will try to add eval support for instruct fine-tuned models soon. Hope this clarifies the confusion.