LAVIS
LAVIS copied to clipboard
Suppress missing key warning when loading partial model weights in BLIP2
ViT (if not finetuned) and LLM weights are not stored in the current BLIP2 checkpoints.
The default load_checkpoint() in BaseModel raises warning on missing keys, which causes confusion.
Proposed solution is to overwrite load_checkpoint() in BLIP2. This solution is not meant to be universal and it shouldn't in general to avoid neglecting missing key by accident.