Katherine Tian

Resume training from checkpoint

1

When I try to feed in a `checkpoint_best.pth` from a previous run to resume training from checkpoint, I get an error at this line (https://github.com/salesforce/LAVIS/blob/main/lavis/runners/runner_base.py#L628) on missing keys in state_dict....

Minor degenerate edge case when resuming training

1

If we run training resuming from epoch N and max epoch is

Blip2 vicuna instruct

3

Do you have a train config for blip2 vicuna instruct? Currently, using a vqa dataset with "blip_question" text processors and a vqa task, I encounter an error at this line...

Unable to train with "clip_L" VIT option (or other generic VIT)

1

I tried to train with clip_L vision encoder, (by adding vit_model: "clip_L" to model train config), but it seems the QFormer checkpoint loaded by default at this line (https://github.com/salesforce/LAVIS/blob/main/lavis/models/base_model.py#L100) is...