EasyEdit
EasyEdit copied to clipboard
Build-in validation in MultimodalTrainer and evaluation in MultimodalEditor
I notice that there is a validate() function in MultimodalTrainer and you validate the trained model after training. Since the validate() function edits the model with the validation set and returns metrics, what's the difference between it and the MultimodalEditor.edit_dataset function, which also edits the model with the validation set and returns metrics. It seems like that thay play the same role. But I get different values of these metrics when I conduct the two functions.
You forget to set new_model.train(False) in SERAC_MULTI.edit() when conducting MultimodalEditor.edit_dataset(), which may be a bug and the reason to produce inconsistent results.