composer icon indicating copy to clipboard operation
composer copied to clipboard

Published 20 hours ago • verified publisher icon mosaicml

mean_resizing = True does not work with mixed/meta initialization

Open jdchang1 opened this issue 11 months ago • 0 comments

What does this PR do?

Transformers recently added in mean_resizing to resize_token_embeddings. This is breaking with mixed initialization in downstream training tasks that requires adding tokens to Composer Huggingface Models. This PR sets this value to False for now rather than defaulting to True.

jdchang1 avatar Nov 20 '24 19:11 jdchang1