mage Are cls_token and pos_embed not used in models

Are cls_token and pos_embed not used in models_mage.py?

Open yxchng opened this issue 1 year ago • 3 comments

https://github.com/LTH14/mage/blob/main/models_mage.py#L188-L189

cls_token and pos_embed are defined here but they don't seem to be used?

Aug 12 '23 11:08 yxchng

Yes, they are deprecated here.

Aug 30 '23 20:08 LTH14

Yes, they are deprecated here.

why pos_embed is not used? I see it is used in MAE.

Nov 21 '23 06:11 xinyuesherry

We use a BertEmb which include its own positional embedding https://github.com/LTH14/mage/blob/main/models_mage.py#L96

Nov 21 '23 12:11 LTH14