Documentation on the differences between the different models

[eduardo4jesus]

Currently, there are three model type available.

• default or vit_h: ViT-H SAM model.
• vit_l: ViT-L SAM model.
• vit_b: ViT-B SAM model.

I could not find any documentation on the difference between them. Is there any available? If not, could someone elaborate on that?

Many thanks.

[franchesoni]

There is a paper accompanying the repository. The models are the same except for neural network size, B stands for "base" and is the smallest, L is "large" and H is "huge". The paper reports that the performance difference between L and H isn't much and I would recommend L if your machine supports it. However, B is lighter and not far behind in performance.

[eduardo4jesus]

@franchesoni, thank you so much. I added a PR #300 on this. I would appreciate to have your feedback.

[pinksloyd]

I've run extensive testing on the models using a wide variety of images. Here is a part of the print-log used when testing and a sample image (locally on my RTX3080):

vit_h Registering model... 12:48:03 Reading image... 12:48:08 Making masks... 12:48:08 Done at: 12:48:14 | Amount: 13 Making image from mask... 12:48:14 Done...? | 12:48:17 | Time taken: 10.918561458587646 vit_l Registering model... 12:48:17 Reading image... 12:48:19 Making masks... 12:48:19 Done at: 12:48:22 | Amount: 17 Making image from mask... 12:48:22 Done...? | 12:48:25 | Time taken: 5.4358086585998535 vit_b Registering model... 12:48:25 Reading image... 12:48:26 Making masks... 12:48:26 Done at: 12:48:28 | Amount: 10 Making image from mask... 12:48:28 Done...? | 12:48:31 | Time taken: 2.9744369983673096

I found that on average vit_l has the best performance/accuracy-tradeoff. vit_h is the most accurate but slowest, and vit_b the fastest but the least accurate.