raghavanone

@sgugger @pacman100 Need pointer on why this test is failing.

> The test is a flaky one, don't worry about it. Thanks for iterating, I just have one last comment on the deprecation warning for `fsdp_min_num_params` and we can merge...

@sgugger @pacman100 Can we merge this PR ?

> ?Hello @raghavanone , could you please resolve the comments above that I have unresolved as they are yet to be addressed ? Done

@muellerzr I would like to pick up this issue and fix it, Looking to write a failing testcase for this bug, Any pointers ?

> Modelling code looks good @raghavanone! Nice one on getting this working so quickly 🙌 Do you want to have a go at adding the encoder-only tests? See the PyTorch...

@sanchit-gandhi There are 2 test failing here, I am unable to get the same failure locally in my machine. Any pointers on how to replicate failing test and fix it...

I will start working on adding the model !

> Hi @MetaB0y , All I have done is to pull in different modules needed for beit3 into single file. I will start working on cleaning it up.

> Hi @raghavanone, just wanted to know updates on this PR. If required, I would like to help. @atharvakavitkar You can for sure contribute, I am out till mid of...