kondvit

> can you give some examples for how to modified code to finetuning a fraction of layers. I would assume you will have to play with this snippet: https://github.com/salesforce/LAVIS/blob/47e0f3f25ca763975738c7224c8369207812ce6c/lavis/models/blip2_models/blip2_opt.py#L60-L65

``I did the following to condition Q-Former. It might not be correct but it is a start. 1. Commented out these lines https://github.com/salesforce/LAVIS/blob/47e0f3f25ca763975738c7224c8369207812ce6c/lavis/models/blip2_models/blip2_t5.py#L75-L79 2. Replaced this code in forward() https://github.com/salesforce/LAVIS/blob/47e0f3f25ca763975738c7224c8369207812ce6c/lavis/models/blip2_models/blip2_t5.py#L112-L117...

I had some experience with 3090s. Definitely can load and train at least OPT6.7 or FLANXL with small batch_size and accum_grad_iters > 1