pytorch-image-models icon indicating copy to clipboard operation
pytorch-image-models copied to clipboard

[FEATURE] add t2t_vit

Open holderhe opened this issue 11 months ago • 1 comments

add t2t_vit

holderhe avatar Dec 14 '24 05:12 holderhe

@holderhe at this point I don't have the time to prioritize models like this, if someone created a fully functional and clean PR I'd consider, but it's a bit of work to pass the tests.

For reference, much closer to original vit arch (with a few tweaks), updated training recipe my explorations here appear quite a bit better than those models on in1k pretrain when you compare img/sec .... see the 82.5 top-1% little vs the 81.5 t2t_vit_14, without f.sdpa the vit_little is slightly faster in infer, quite a bit in train, and then with the F.sdpa (default0 enabled it's much faster. https://huggingface.co/timm/vit_little_patch16_reg1_gap_256.sbb_in12k

rwightman avatar Dec 14 '24 22:12 rwightman