[BLIP2] lavis/tasks/base_task.py accum_grad_iters division for reduction

[kondvit]

Hello,

First of all, awesome work Salesforce team.

I am finetuning Blip2FlanXL

I noticed in base_task.py you are not dividing loss by accum_grad_iters.

https://github.com/salesforce/LAVIS/blob/47e0f3f25ca763975738c7224c8369207812ce6c/lavis/tasks/base_task.py#L214-L230

For FlanXL the default loss reduction is 'mean'.

https://github.com/salesforce/LAVIS/blob/47e0f3f25ca763975738c7224c8369207812ce6c/lavis/models/blip2_models/modeling_t5.py#L1752

In case of FlanXL with default mean loss reduction, should I divide the loss by accum_grad_iters?

https://github.com/salesforce/LAVIS/blob/47e0f3f25ca763975738c7224c8369207812ce6c/lavis/tasks/base_task.py#L215

to

loss = self.train_step(model=model, samples=samples) / accum_grad_iters

Thank you