Tweaks for lm-eval-harness

[zphang]

trafficstars

• Importing one fix from Microsoft/DeepSpeed
• Disable sanity_check, which does not seem to be doing the right thing. (For some reason it checks against every possible mergeable weight key, for every layer)

[microsoft-cla-retired[bot]]

Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

:x: zphang sign now
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[rocm-mici]

Can one of the admins verify this patch?

[mrwyattii]

This PR is quite old and it appears outdated. Looks like the relevant changes occurred here https://github.com/bigscience-workshop/Megatron-DeepSpeed/pull/212