mergekit icon indicating copy to clipboard operation
mergekit copied to clipboard

Published 20 hours ago • verified publisher icon arcee-ai

Merge only the transformer parts (including the input embedding layer)

Open hank0316 opened this issue 9 months ago • 5 comments

Hi,

I want to know how to merge the transformer layers and input embedding layer only. I want to keep the original lm_head. Is this possible?

hank0316 avatar May 08 '24 12:05 hank0316