Clarification of "Model used in filtering"

[fajri91]

Hi guys,

Is this the resulting model of the pretraining step (i.e. prior to fine-tuning)?

[procedure2012]

Yes, the weights are from full models, which are pretrained with all pretraining tasks.