MI-Zero
MI-Zero copied to clipboard
Invalid dimension of transformer.embeddings.word_embeddings.weight for the ctranspath_448_bioclinicalbert checkpoint
RuntimeError: Error(s) in loading state_dict for CLIP:
size mismatch for transformer.embeddings.word_embeddings.weight: copying a param with shape torch.Size([30522, 768]) from checkpoint, the shape in current model is torch.Size([28996, 768]).
The dimension of bioclinicalbert emilyalsentzer/Bio_ClinicalBERT
(whose vocab size is 28996, and the vocab size of pubmedbert is 30522) and the provided ctranspath_448_bioclinicalbert.zip checkpoint are not the same, and causes failure in loading the checkpoint.