oneflow icon indicating copy to clipboard operation
oneflow copied to clipboard

Published 20 hours ago • verified publisher icon Oneflow-Inc

Speed up the training

Open Yipeng1994 opened this issue 3 years ago • 0 comments

Mentioned in https://github.com/Oneflow-Inc/OneTeam/issues/1735, some operators might need to be run as late as possible since they have a large activation time in cpu. In this feature, we move those operators backward and reduce the idle time in cuda by 40% (11.5ms -> 7ms per iteration)

Currently no obvious speeding up

Yipeng1994 avatar Oct 20 '22 09:10 Yipeng1994