BHUVAN M

Just changing the input shape in the nn.linear region to 49 should solve the problem ig..

i can work on vit_mae and tvp

Since all clip like models can just borrow changes made to clip model, I will take tvp instead of altclip.

Apologies, My previous PR got polluted with commits due to my negligence and lack of knowledge :( . I will make sure that is not repeated ever again.

Done @amyeroberts

I think by default the tensors are created in CPU and you have used only ```torch.cuda.manual_seed``` and not ```torch.manual_seed```. so the tensors created have become random. Try it once and...

@amyeroberts should I sync this branch wrt to upstream main branch?