torchrec icon indicating copy to clipboard operation
torchrec copied to clipboard

Published 20 hours ago • verified publisher icon pytorch

Log dense storage and kjt storage for APS training jobs

Open shourya-mukherjee opened this issue 1 year ago • 1 comments

Summary: We parse torchrec logs from individual jobs (e.g. P1542961580). Our analyzer fails when Dense/KJT Storage is not available in the logs. This can be due to different types of storage reservations. We should attempt to extract dense and kjt storage to logs regardless of the type of storage reservation.

Differential Revision: D63666701

shourya-mukherjee avatar Oct 05 '24 01:10 shourya-mukherjee