raghavanone

@ArthurZucker I can pick up this, Let me know what should be possible fix ?

@sgugger @NielsRogge The model weights are here https://huggingface.co/ZinengTang/Udop/tree/main , But how to get the config for these models ?

Note: Cannot proceed further without microsoft releasing the entire weights. Currently vision decoder weights have not been released.

@logan-markewich Yes, I will work on closing this within couple of days .

@sgugger Need some pointers on How should this model be tested ? Can I follow the tests used for T5 model and replicate similar tests ?

@NielsRogge Any pointer here ?

> @raghavanone how are things going with this so far? I'm very interested in using this model as soon as it gets integrated - if you need a hand with...

> @raghavanone I am currently trying to finetune `UdopUniModelForConditionalGeneration` using this PR. I ran into the following exception while training: > > ``` > File "/opt/conda/lib/python3.8/site-packages/transformers/models/udop/modeling_udop.py", line 2422, in forward...

@maxjeblick @NielsRogge feels that the code original repo is bit hacky, he is working a separate PR to UDOP in better implementation, so closed this in consultation with him. He...

@sgugger Done, But not sure why this test is failing. Any pointers on how to make this build green would help.