oslo icon indicating copy to clipboard operation
oslo copied to clipboard

Published 20 hours ago • verified publisher icon EleutherAI

Fix TP embedding layers

Open jason9693 opened this issue 1 year ago • 0 comments

Describe a TODO feature

  • Force tp_wrapper do not parallelize emb-layer if model has not embedding layer. (for vision model competible) https://discord.com/channels/729741769192767510/1012603449910759504/1083785802930192434

Assignees

  • @jason9693

jason9693 avatar Mar 12 '23 06:03 jason9693