Diffusion-BERT icon indicating copy to clipboard operation
Diffusion-BERT copied to clipboard
verified publisher icon Hzfinfdu

How to fine-tune it

Open bansky-cl opened this issue 2 years ago • 1 comments

It seems that a pre-trained language model. Could i run train on a lot of unconditional text to get a checkpoint then fine-tuning the model on Seq2seq tasks?

bansky-cl avatar May 28 '23 14:05 bansky-cl

Sure! (Continue) pre-training and finetuning with the diffusion objective are both supported.

Hzfinfdu avatar May 30 '23 15:05 Hzfinfdu