Config problem

[KerwenX]

hello, @patricklabatut,

when i run the code in semantic_segmentation.ipynb, I got this error. I didn't see any definitions about VisionTransformerLS in mmseg, so I would like to ask how I can solve this problem. Thank you.

code:

import math
import itertools
from functools import partial

import torch
import torch.nn.functional as F
from mmseg.apis import init_model, inference_model
import sys
sys.path.append('/home/aston/Desktop/project_new/dinov2')
import dinov2.eval.segmentation.models


class CenterPadding(torch.nn.Module):
    def __init__(self, multiple):
        super().__init__()
        self.multiple = multiple

    def _get_pad(self, size):
        new_size = math.ceil(size / self.multiple) * self.multiple
        pad_size = new_size - size
        pad_size_left = pad_size // 2
        pad_size_right = pad_size - pad_size_left
        return pad_size_left, pad_size_right

    @torch.inference_mode()
    def forward(self, x):
        pads = list(itertools.chain.from_iterable(self._get_pad(m) for m in x.shape[:1:-1]))
        output = F.pad(x, pads)
        return output


def create_segmenter(cfg, backbone_model):
    model = init_model(cfg)
    model.backbone.forward = partial(
        backbone_model.get_intermediate_layers,
        n=cfg.model.backbone.out_indices,
        reshape=True,
    )
    if hasattr(backbone_model, "patch_size"):
        model.backbone.register_forward_pre_hook(lambda _, x: CenterPadding(backbone_model.patch_size)(x[0]))
    model.init_weights()
    return model

import urllib

import mmengine
from mmengine.runner import load_checkpoint


def load_config_from_url(url: str) -> str:
    with urllib.request.urlopen(url) as f:
        return f.read().decode()


HEAD_SCALE_COUNT = 3 # more scales: slower but better results, in (1,2,3,4,5)
HEAD_DATASET = "ade20k" # in ("ade20k", "voc2012")
HEAD_TYPE = "ms" # in ("ms, "linear")


DINOV2_BASE_URL = "https://dl.fbaipublicfiles.com/dinov2"
head_config_url = f"{DINOV2_BASE_URL}/{backbone_name}/{backbone_name}_{HEAD_DATASET}_{HEAD_TYPE}_config.py"
head_checkpoint_url = f"{DINOV2_BASE_URL}/{backbone_name}/{backbone_name}_{HEAD_DATASET}_{HEAD_TYPE}_head.pth"

cfg_str = load_config_from_url(head_config_url)


cfg = mmengine.config.Config.fromstring(cfg_str, file_format=".py")
if HEAD_TYPE == "ms":
    cfg.data.test.pipeline[1]["img_ratios"] = cfg.data.test.pipeline[1]["img_ratios"][:HEAD_SCALE_COUNT]
    print("scales:", cfg.data.test.pipeline[1]["img_ratios"])

# print(cfg)
model = create_segmenter(cfg, backbone_model=backbone_model)
load_checkpoint(model, head_checkpoint_url, map_location="cpu")
model.cuda()
model.eval()

output:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[3], line 35
     32     print("scales:", cfg.data.test.pipeline[1]["img_ratios"])
     34 # print(cfg)
---> 35 model = create_segmenter(cfg, backbone_model=backbone_model)
     36 load_checkpoint(model, head_checkpoint_url, map_location="cpu")
     37 model.cuda()

Cell In[1], line 33, in create_segmenter(cfg, backbone_model)
     32 def create_segmenter(cfg, backbone_model):
---> 33     model = init_model(cfg)
     34     model.backbone.forward = partial(
     35         backbone_model.get_intermediate_layers,
     36         n=cfg.model.backbone.out_indices,
     37         reshape=True,
     38     )
     39     if hasattr(backbone_model, "patch_size"):

File [~/anaconda3/envs/dinov2-extras/lib/python3.9/site-packages/mmseg/apis/inference.py:54](https://file+.vscode-resource.vscode-cdn.net/home/aston/Desktop/project_new/dinov2/notebooks/~/anaconda3/envs/dinov2-extras/lib/python3.9/site-packages/mmseg/apis/inference.py:54), in init_model(config, checkpoint, device, cfg_options)
     51 config.model.train_cfg = None
     52 init_default_scope(config.get('default_scope', 'mmseg'))
---> 54 model = MODELS.build(config.model)
     55 if checkpoint is not None:
     56     checkpoint = load_checkpoint(model, checkpoint, map_location='cpu')
...
    106         )
    107 # this will include classes, functions, partial functions and more
    108 elif callable(obj_type):

KeyError: 'VisionTransformerLS is not in the mmseg::model registry. Please check whether the value of `VisionTransformerLS` is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module'

[vijayreddysamula]

Hi @KerwenX From my understanding the provided config file is only VOC2012 dataset HEAD_DATASET = "ade20k" . They also provide config file for mask2former in the last part.

[KerwenX]

Hi @vijayreddysamula I understand your viewpoint, but I believe the pre-trained model of HEAD_DATASET="ade20k" provided for the ADE20k dataset may be a bit confusing.

[patricklabatut]

@KerwenX Thanks for reporting, it looks like the configs were not properly synced. This should (hopefully) be fixed now, but you might have to clear the PyTorch Hub cache.

[KerwenX]

Hi, @patricklabatut Additionally, when i run the code in the block Load pretrained segmentation model (Mask2Former) of semantic_segmentation.ipynb, I got the same question as above. KeyError: 'EncoderDecoderMask2Former is not in the models registry'

code:

import sys
sys.path.append('/home/aston/Desktop/project_new/dinov2/')
# import dinov2.eval.segmentation_m2f.models.segmentors

CONFIG_URL = f"{DINOV2_BASE_URL}/dinov2_vitg14/dinov2_vitg14_ade20k_m2f_config.py"
CHECKPOINT_URL = f"{DINOV2_BASE_URL}/dinov2_vitg14/dinov2_vitg14_ade20k_m2f.pth"

cfg_str = load_config_from_url(CONFIG_URL)
cfg = mmcv.Config.fromstring(cfg_str, file_format=".py")

model = init_segmentor(cfg)
load_checkpoint(model, CHECKPOINT_URL, map_location="cpu")
model.cuda()
model.eval()

output:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[11], line 11
      8 cfg_str = load_config_from_url(CONFIG_URL)
      9 cfg = mmcv.Config.fromstring(cfg_str, file_format=".py")
---> 11 model = init_segmentor(cfg)
     12 load_checkpoint(model, CHECKPOINT_URL, map_location="cpu")
     13 model.cuda()

File [~/anaconda3/envs/dinov2-extras/lib/python3.9/site-packages/mmseg/apis/inference.py:32](https://file+.vscode-resource.vscode-cdn.net/home/aston/Desktop/project_new/dinov2/notebooks/~/anaconda3/envs/dinov2-extras/lib/python3.9/site-packages/mmseg/apis/inference.py:32), in init_segmentor(config, checkpoint, device)
     30 config.model.pretrained = None
     31 config.model.train_cfg = None
---> 32 model = build_segmentor(config.model, test_cfg=config.get('test_cfg'))
     33 if checkpoint is not None:
     34     checkpoint = load_checkpoint(model, checkpoint, map_location='cpu')

File [~/anaconda3/envs/dinov2-extras/lib/python3.9/site-packages/mmseg/models/builder.py:48](https://file+.vscode-resource.vscode-cdn.net/home/aston/Desktop/project_new/dinov2/notebooks/~/anaconda3/envs/dinov2-extras/lib/python3.9/site-packages/mmseg/models/builder.py:48), in build_segmentor(cfg, train_cfg, test_cfg)
     44 assert cfg.get('train_cfg') is None or train_cfg is None, \
     45     'train_cfg specified in both outer field and model field '
     46 assert cfg.get('test_cfg') is None or test_cfg is None, \
     47     'test_cfg specified in both outer field and model field '
---> 48 return SEGMENTORS.build(
     49     cfg, default_args=dict(train_cfg=train_cfg, test_cfg=test_cfg))

File [~/anaconda3/envs/dinov2-extras/lib/python3.9/site-packages/mmcv/utils/registry.py:215](https://file+.vscode-resource.vscode-cdn.net/home/aston/Desktop/project_new/dinov2/notebooks/~/anaconda3/envs/dinov2-extras/lib/python3.9/site-packages/mmcv/utils/registry.py:215), in Registry.build(self, *args, **kwargs)
    214 def build(self, *args, **kwargs):
--> 215     return self.build_func(*args, **kwargs, registry=self)

File [~/anaconda3/envs/dinov2-extras/lib/python3.9/site-packages/mmcv/cnn/builder.py:27](https://file+.vscode-resource.vscode-cdn.net/home/aston/Desktop/project_new/dinov2/notebooks/~/anaconda3/envs/dinov2-extras/lib/python3.9/site-packages/mmcv/cnn/builder.py:27), in build_model_from_cfg(cfg, registry, default_args)
     25     return Sequential(*modules)
     26 else:
---> 27     return build_from_cfg(cfg, registry, default_args)

File [~/anaconda3/envs/dinov2-extras/lib/python3.9/site-packages/mmcv/utils/registry.py:44](https://file+.vscode-resource.vscode-cdn.net/home/aston/Desktop/project_new/dinov2/notebooks/~/anaconda3/envs/dinov2-extras/lib/python3.9/site-packages/mmcv/utils/registry.py:44), in build_from_cfg(cfg, registry, default_args)
     42     obj_cls = registry.get(obj_type)
     43     if obj_cls is None:
---> 44         raise KeyError(
     45             f'{obj_type} is not in the {registry.name} registry')
     46 elif inspect.isclass(obj_type):
     47     obj_cls = obj_type

KeyError: 'EncoderDecoderMask2Former is not in the models registry'

[qasfb]

Is there a specific reason why there is a commented import in the code ?

[patricklabatut]

@qasfb is right about the last piece of code, some imports of the notebooks (and specifically dinov2.eval.segmentation_m2f.models.segmentors here) are actually required to properly register the supporting modules for the model. So removing them could lead to the error you see.

[lumalav]

I had to do the same thing @KerwenX did on a cloned google colab because I was getting a:

ModuleNotFoundError: No module named 'dinov2.eval.segmentation_m2f.ops'

But, now I get a similar error: KeyError: "EncoderDecoderMask2Former: 'ViTAdapter is not in the models registry'"

import sys
sys.path.append("/content/dinov2")

CONFIG_URL = f"{DINOV2_BASE_URL}/dinov2_vitg14/dinov2_vitg14_ade20k_m2f_config.py"
CHECKPOINT_URL = f"{DINOV2_BASE_URL}/dinov2_vitg14/dinov2_vitg14_ade20k_m2f.pth"

cfg_str = load_config_from_url(CONFIG_URL)
cfg = mmcv.Config.fromstring(cfg_str, file_format=".py")
model = init_segmentor(cfg)
load_checkpoint(model, CHECKPOINT_URL, map_location="cpu")
model.eval()
model.cuda()`

`---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/mmcv/utils/registry.py](https://localhost:8080/#) in build_from_cfg(cfg, registry, default_args)
     51     try:
---> 52         return obj_cls(**args)
     53     except Exception as e:

10 frames
KeyError: 'ViTAdapter is not in the models registry'

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/mmcv/utils/registry.py](https://localhost:8080/#) in build_from_cfg(cfg, registry, default_args)
     53     except Exception as e:
     54         # Normal TypeError does not print class name.
---> 55         raise type(e)(f'{obj_cls.__name__}: {e}')
     56 
     57 

KeyError: "EncoderDecoderMask2Former: 'ViTAdapter is not in the models registry'"