[not tested] Fix encoder conversion by using base model

[kabachuha]

trafficstars

Using existing clip checkpoint in modelscope format change the trained layers, so it will maintain integrity and not fail to load