pre-training code tutorial

[Akhilesh64]

Hi I was wondering if you could prepare a tutorial for pre-training DocFormer from scratch. It would be a huge help.

[uakarsh]

Hi @Akhilesh64 , thanks a lot for your request.

I am preparing the end to end pre-training and finetuning code here: https://github.com/uakarsh/docformer/tree/master/examples/docformer_pl/docformer_etoe_pretraining_finetuning

[Akhilesh64]

@uakarsh I am confused should I use your link or this one https://github.com/shabie/docformer/blob/master/examples/docformer_pl/pre_training_task_on_idl_dataset/docformer_pre_training_3_modeling_for_docformer.py

[uakarsh]

It was an old script. Although I believe it is correct, but the link I shared is the one which we are planning to work on, to pre-train docformer