The prompt appears when the initializing checkpoint appears "This is not exten if you are initializing cliptextModel ..."

[xiaoyoucheng]

- This IS expected if you are initializing CLIPTextModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceCla
ssification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing CLIPTextModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification mo
del from a BertForSequenceClassification model).

I'm not sure if I need to change anything to make it disappear

[tyz1994]

I also meet the same issue, saying "vision_model.xx not used". I believe it will not affect the training, as vision model in CLIP is never used in the text encoder. This should be fixed in transformers i think.