torchtitan icon indicating copy to clipboard operation
torchtitan copied to clipboard

Published 20 hours ago • verified publisher icon pytorch

add pipeline parallelism to train llama

Open wanchaol opened this issue 1 year ago • 1 comments

wanchaol avatar Jan 23 '24 20:01 wanchaol

I will start a sep branch for pippy integration and begin testing with this.

lessw2020 avatar Jan 27 '24 20:01 lessw2020

3D parallel enabled in #344

tianyu-l avatar Jun 12 '24 23:06 tianyu-l