IJCAI2023-OptimalShardedDataParallel icon indicating copy to clipboard operation
IJCAI2023-OptimalShardedDataParallel copied to clipboard

Published 20 hours ago • verified publisher icon Youhe-Jiang

pytorch version

Open tisgotos opened this issue 1 year ago • 2 comments
trafficstars

Hello! I have obtained a ViT model from timm, and I want to train it using your OSDP method. However, OSDP requires torch version 1.10.2, while timm needs a higher version. What should I do in this situation?

tisgotos avatar Jul 23 '24 12:07 tisgotos

Actually we have merged the function of OSDP to Galvatron from Hetu, also a high performance training framework, you can check the newest release version Galvatron-2 for optimized implementation. https://github.com/PKU-DAIR/Hetu-Galvatron

Youhe-Jiang avatar Jul 23 '24 13:07 Youhe-Jiang

Actually we have merged the function of OSDP to Galvatron from Hetu, also a high performance training framework, you can check the newest release version Galvatron-2 for optimized implementation. https://github.com/PKU-DAIR/Hetu-Galvatron

Thank you!

tisgotos avatar Jul 24 '24 01:07 tisgotos