Error Loading Pre-trained Weights: Size Mismatch in DPTDepthModel when trying to run for first time.

[pablobiedma]

Hello everyone, I am new to MiDaS and I am trying to get it to run for the first time. I followed the instructions and added the downloaded weights to the weight folder. Then I added 10 random images to the input folder. After I run execution the following command: python run.py --model_type dpt_swin2_tiny_256 --input input --output output (also with model and weights for dpt_beit_large_512 and dpt_swin2_large_384) I get the following error message:

Initialize

Device: cpu
C:\Users\Pablo\anaconda3\lib\site-packages\torch\functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\cb\pytorch_1000000000000\work\aten\src\ATen\native\TensorShape.cpp:3484.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
Traceback (most recent call last):
  File "C:\Users\Pablo\desktop\midas\run.py", line 276, in <module>
    run(args.input_path, args.output_path, args.model_weights, args.model_type, args.optimize, args.side, args.height,
  File "C:\Users\Pablo\desktop\midas\run.py", line 126, in run
    model, transform, net_w, net_h = load_model(device, model_path, model_type, optimize, height, square)
  File "C:\Users\Pablo\desktop\midas\midas\model_loader.py", line 102, in load_model
    model = DPTDepthModel(
  File "C:\Users\Pablo\desktop\midas\midas\dpt_depth.py", line 163, in __init__
    self.load(path)
  File "C:\Users\Pablo\desktop\midas\midas\base_model.py", line 16, in load
    self.load_state_dict(parameters)
  File "C:\Users\Pablo\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for DPTDepthModel:
        Missing key(s) in state_dict: "pretrained.model.layers.3.downsample.reduction.weight", "pretrained.model.layers.3.downsample.norm.weight", "pretrained.model.layers.3.downsample.norm.bias", "pretrained.model.head.fc.weight", "pretrained.model.head.fc.bias".
        Unexpected key(s) in state_dict: "pretrained.model.layers.0.downsample.reduction.weight", "pretrained.model.layers.0.downsample.norm.weight", "pretrained.model.layers.0.downsample.norm.bias", "pretrained.model.layers.0.blocks.1.attn_mask", "pretrained.model.layers.1.blocks.1.attn_mask", "pretrained.model.head.weight", "pretrained.model.head.bias".
        size mismatch for pretrained.model.layers.1.downsample.reduction.weight: copying a param with shape torch.Size([384, 768]) from checkpoint, the shape in current model is torch.Size([192, 384]).
        size mismatch for pretrained.model.layers.1.downsample.norm.weight: copying a param with shape torch.Size([384]) from checkpoint, the shape in current model is torch.Size([192]).
        size mismatch for pretrained.model.layers.1.downsample.norm.bias: copying a param with shape torch.Size([384]) from checkpoint, the shape in current model is torch.Size([192]).
        size mismatch for pretrained.model.layers.2.downsample.reduction.weight: copying a param with shape torch.Size([768, 1536]) from checkpoint, the shape in current model is torch.Size([384, 768]).
        size mismatch for pretrained.model.layers.2.downsample.norm.weight: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([384]).
        size mismatch for pretrained.model.layers.2.downsample.norm.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([384]).

Essentially, I am encountering a size mismatch error when trying to load pre-trained weights into the DPTDepthModel. Despite ensuring that the model type and the pre-trained weights should be compatible, the error persists.

Do you know what my mistake is? Any help is appreciated

[weijielyu]

Same problem here

[pablobiedma]

@weijielyu I have managed to circumvent this issue by computing Depth of Field directly using this script, feel free to take a look: https://github.com/pablobiedma/DoF-with-MiDaSv3.1

[weijielyu]

@weijielyu I have managed to circumvent this issue by computing Depth of Field directly using this script, feel free to take a look: https://github.com/pablobiedma/DoF-with-MiDaSv3.1

Got it, thank you!

[uglymie]

@weijielyu I have managed to circumvent this issue by computing Depth of Field directly using this script, feel free to take a look: https://github.com/pablobiedma/DoF-with-MiDaSv3.1 Your code does not load the model trained by MiDaSv3.1, but MiDaS v2.1

[uglymie]

If the version of timm is too high, it will cause this problem. I replaced timm 0.9.7 with version 0.6.12 adapted to the project, and the problem disappeared

[nikky4D]

If the version of timm is too high, it will cause this problem. I replaced timm 0.9.7 with version 0.6.12 adapted to the project, and the problem disappeared

This solved the issue for me

[NielsRogge]

For easy inference see #259