Loading Mask2Former with the newest mmcv

[julkaztwittera]

Is there any way to load Dinov2 with Mask2Former for instance segmentation having the newest mmcv-full (instead of 1.5.0) installed?

[dillonalaird]

I found it easier to use the DINOv2 model provided in mmpretrain here. You can load it with something like the following config:

pretrained = 'https://download.openmmlab.com/mmpretrain/v1.0/dinov2/vit-small-p14_dinov2-pre_3rdparty_20230426-5641ca5a.pth'

model = dict(
    type='EncoderDecoder',
    backbone=dict(
        type='mmpretrain.VisionTransformer',
        arch='dinov2-small',
        img_size=518,
        patch_size=14,
        out_indices=(8, 9, 10, 11),
        frozen_stages=-1,
        init_cfg=dict(type='Pretrained', checkpoint=pretrained, prefix='backbone.')
    ),
    decode_head=dict(
        type='Mask2FormerHead',
        in_channels=[384, 384, 384, 384],
        ...
    ),
)

[Atumyugi]

I found it easier to use the DINOv2 model provided in mmpretrain here. You can load it with something like the following config:

pretrained = 'https://download.openmmlab.com/mmpretrain/v1.0/dinov2/vit-small-p14_dinov2-pre_3rdparty_20230426-5641ca5a.pth'

model = dict(
    type='EncoderDecoder',
    backbone=dict(
        type='mmpretrain.VisionTransformer',
        arch='dinov2-small',
        img_size=518,
        patch_size=14,
        out_indices=(8, 9, 10, 11),
        frozen_stages=-1,
        init_cfg=dict(type='Pretrained', checkpoint=pretrained, prefix='backbone.')
    ),
    decode_head=dict(
        type='Mask2FormerHead',
        in_channels=[384, 384, 384, 384],
        ...
    ),
)

hi, Could you please provide more specific code and configuration files? Thank you very much 😀

[GLASS-z13]

does anybody have an example of using dinov2 with nask2former as a head? (without mmcv)