Akarsh

I would be working from my side for MLM (although there are 3 pre-training task) and would update shortly. Thanks,

Hi @jmandivarapu1 @AYSP can you guys again try the fine-tuning using the pre-trained weights (I have attached them in the readme)

Hi @Arun-purakkatt, if that's the case, I think here is what we can do 1. See how the `validation_epoch_end` is actually implemented for `pytorch_lightning v2`, and then just rename the...

@shabie , I am planning to do it :). Quite excited to do it. as far as I see, more of the details is in pre-training, so to release weights,...

Not sure, how to go about it, because I tried just a few samples of documents. I think, there is something which saturates the entire training, would look for it,...

Can you let me know, what approach did you try? I guess, it would be helpful for me as well, to conduct the experiment

Got it. Thanks, and would update here, as soon as possible

Hi @HoomanKhosravi, I did try to train the docformer on the FUNSD dataset from scratch. Here is the script, [Kaggle Notebook](https://www.kaggle.com/akarshu121/docformer-for-token-classification-on-funsd), and here are the results [Weight and Bias Report](https://wandb.ai/iakarshu/DocFormer%20on%20FUNSD?workspace=user-iakarshu)...

Hi, nope, I haven't tried to pre-train and then fine-tune it, but I am planning to do it in a few days and then observe the results. I guess, I...

I guess pre-training would help, I have managed to prepare a small data for pre-training. It is actually IDL Dataset (https://github.com/furkanbiten/idl_data) and in some time I am about to train...