devit icon indicating copy to clipboard operation
devit copied to clipboard

Published 20 hours ago • verified publisher icon mlzxy

Vit model weights

Open gautham-98 opened this issue 9 months ago • 0 comments

  1. Why does build_prototypes.ipynb use model = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitl14') while inference in demo.py uses model_path="weights/trained/open-vocabulary/lvis/vitl_0069999.pth"? Should'nt the models be the same ?

  2. Why do we have different weights file for COCO and LVIS? as per my understanding the Vit model remains the same which is DinoV2 even if the datasets changes, since there is no fine tuning.

  3. Finally for a custom dataset if there is no domain gap the steps would be to 1. create prototypes, 2. Run the demo, and if there is a domain gap i guess we have to fine tune the RPN and leave the Vits part as it is. Would this be the way?

gautham-98 avatar May 16 '24 09:05 gautham-98