Mismatch between weights and config file.

[magnusvan]

I get the following warnings while using the wholebody weights for tcformer large:

size mismatch for backbone.blocks.0.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.0.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.1.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.1.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.2.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.2.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.3.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.3.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.4.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.4.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.5.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.5.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.6.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.6.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.7.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.7.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.8.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.8.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.9.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.9.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.10.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.10.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.11.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.11.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.12.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.12.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.13.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.13.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.14.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.14.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.15.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.15.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.16.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.16.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.17.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.17.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.18.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.18.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.19.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.19.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.20.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.20.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.21.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.21.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.22.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.22.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.23.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.23.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.24.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.24.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.25.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.25.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.26.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.26.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.27.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.27.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.28.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.28.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.29.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.29.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.30.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.30.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for backbone.blocks.31.mlp.fc2.weight: copying a param with shape torch.Size([960, 5120]) from checkpoint, the shape in current model is torch.Size([1280, 5120]).
size mismatch for backbone.blocks.31.mlp.fc2.bias: copying a param with shape torch.Size([960]) from checkpoint, the shape in current model is torch.Size([1280]).
size mismatch for keypoint_head.final_layer.weight: copying a param with shape torch.Size([17, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([133, 256, 1, 1]).
size mismatch for keypoint_head.final_layer.bias: copying a param with shape torch.Size([17]) from checkpoint, the shape in current model is torch.Size([133]).

It seems that there is a mismatch in terms of channels.

Config file used: pose/configs/wholebody/2d_kpt_sview_rgb_img/topdown_heatmap/coco-wholebody/tcformer_large_mta_coco_wholebody_384x288.py

Link followed: https://drive.google.com/file/d/1aUIj_-U1EfklVGzELUrierwFNoUp-zrH/view?usp=sharing checkpoint downloaded: tcformer_large_coco-wholebody_384x288-b3b884c8_20220627.pth

There is obviously something that i am doing wrong.

COCO Wholebody has 133 keypoints so it seems like the weights have the wrong dimension and only the 17 keypoints.

Can anyone help me point me in the right direction?

Thank you!

[jin-s13]

TCFormer wholebody models are supported in mmpose v0.28.0.