pytorch-image-models icon indicating copy to clipboard operation
pytorch-image-models copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

[FEATURE] BEIT pre-training model

Open lorenzbaraldi opened this issue 2 years ago • 2 comments

Is your feature request related to a problem? Please describe. There is no problem or bug

Describe the solution you'd like I would like the implementation of BEIT pre-training pipeline in order to be able to manually pre-training the architecture

Describe alternatives you've considered No

Additional context No

lorenzbaraldi avatar Dec 23 '22 09:12 lorenzbaraldi

@lorenzbaraldi MIM (BEiT and MAE style) support is on the todo list but requires some careful experimentation, no sure what the timeline is...

rwightman avatar Dec 23 '22 18:12 rwightman

@rwightman since I need to work on that for my research, do you think is it useful to create new training functions (like "train_one_epoch" and validate) that are specific for pre-training phase?

lorenzbaraldi avatar Dec 24 '22 01:12 lorenzbaraldi