allenai
Adding script for processing many intermediate checkpoints at once for offline evals
Making a draft PR for this so we can consider merging this in to main. It would be nice if we could do this so we don't run into version issues if we train models in the future that are not compatible with the version of the code forked here.
@soldni , don't you already have a checkpoint converter script that runs in Beaker?
He does. What this one does is very similar but the focus is on batch conversion and wildcard acceptance. And it was written for oe-eval consistent ranking project with expediting its pipeline in mind.
@jenahwang would it be possible to merge in your changes to the other script? or consolidate the two?
it's confusing to have two conversion scripts, and it doubles maintenance.
