MI-Zero icon indicating copy to clipboard operation
MI-Zero copied to clipboard

Invalid dimension of transformer.embeddings.word_embeddings.weight for the ctranspath_448_bioclinicalbert checkpoint

Open Lafite-Yu opened this issue 4 months ago • 0 comments

RuntimeError: Error(s) in loading state_dict for CLIP:
	size mismatch for transformer.embeddings.word_embeddings.weight: copying a param with shape torch.Size([30522, 768]) from checkpoint, the shape in current model is torch.Size([28996, 768]).

The dimension of bioclinicalbert emilyalsentzer/Bio_ClinicalBERT (whose vocab size is 28996, and the vocab size of pubmedbert is 30522) and the provided ctranspath_448_bioclinicalbert.zip checkpoint are not the same, and causes failure in loading the checkpoint.

Lafite-Yu avatar Oct 11 '24 09:10 Lafite-Yu