[BEATs] difference between 2 AS-finetuned checkpoint versions?

[slSeanWU]

Hi @Sanyuan-Chen, thanks a lot for releasing the implementation and checkpoints of BEATs. It helps a lot with my project.

May I ask is there any fundamental difference (e.g., in hyperparameters) between the two AudioSet finetuned checkpoints offered at each iteration? Or are they just trained with different random seeds?

Screenshot from README:

Thanks! -- Shih-Lun

[Sanyuan-Chen]

Hi @slSeanWU , thanks for your interest in our paper.

The only difference between the two checkpoints is the fine-tuning hyperparameters (learning rate and update steps). We use the default hyperparameters shown in Table 4 for cpt1, and 5e-5 learning rate and 100k training steps for cpt2 as introduced in Section 4.6 of our paper.

[slSeanWU]

Thanks for the clarification @Sanyuan-Chen.

So does cpt2 generally outperform cpt1? I am asking since the paper seemed to only report ensemble model performance. Thanks.

[Sanyuan-Chen]

The cpt1 models have slightly better results. We provide the cpt2 models here for the reproduction of our ensemble results.