ViTAE-Transformer
where the model code?
@yuan243212790 @WT-UCAS @hao14293 take this with a grain of salt cause I just started looking through the repo and the paper. But the backbone is ViT.py and they trained 3 models in the paper based off an image is worth 16x16 words. There are the Base, Large, and Huge models (which means you can modify ViT.py class Vit(BaseBackbone) args for 3/4 parameters that differentiate the Base, Large, and Huge models (hidden size, mlp size, heads all that is missing in layers).
Also, looking through the paper, they seem to use a simple decoder which is probably coming from mmpose.models.heads.deconv_head.py
The actual building of the model seems to be happening from two main places: mmcv (which you install with this repo) and then inside, mmcv.cnn builder.py and teh config file that sets the hyperparameters to construct the model from the ViT and deconv_head can be found in ViTPose/configs/body/2d_kpt_sview_rgb_img/topdown_heamap/coco
Hopefully this helps you get started in your search. I would love to hear from any of the contributors as too how accurate this is. @Annbless @AK391
The different configurations of our models can be found in the folder. In each config file, there is a dictionary named model, which contains two sub-dictionaries named backbone and keypoint head, which define the exact structure of the backbone networks and prediction heads. The structure of the models described using the backbone dictionary can be found in the mmpose/models/backbones/ folders. Similarly, the exact structure of the keypoint head can be found in mmpose/models/heads/.
It seems there are no further questions. I will close this issue temporarily. If you have any further questions, please feel free to re-open it.