pcircle
pcircle copied to clipboard
Optimize fprof for Lustre attribute retrieval
The "lfs getstripe" and "lfs find" code is using an ioctl(LL_IOC_MDC_GETINFO) to fetch both the stat attributes and Lustre layout from the MDS in a single RPC. This also opens only the parent directory for the ioctl(), so that each file does not need to be opened (which avoids inode cache pollution and DLM lock/cancel overhead as well).
With the "Lazy Size-on-MDT" (LSOM) feature that is being added in 2.12 it would also be possible to fetch a reasonable estimate of the file size from the MDS as well, which would avoid the need for any OSS RPCs entirely, which can add significant overhead if the MDT(s) are SSDs and the OSTs are HDDs. For the purposes of fprof (histograms and average file sizes), the LSOM size is sufficient.
thanks for the suggestion, Andreas, I will look into it.
Feiyi