scale input image size problem

[nazimii]

Hi, I meet a problem when try to scale input image size, 1024 is to big so I scale it to 224, but if I use the "use_rel_pos=True", the input image_size args in all ckpt is 1024, it will report error (size mismatch at blocks 2, 5、8、11) at Attention() module in VIT. so, is it feasible to set encoder_global_attn_indexes = [] to avoid this error? Will the performance degrade？