UnifyFS icon indicating copy to clipboard operation
UnifyFS copied to clipboard

Lustre group lock API

Open adammoody opened this issue 3 years ago • 0 comments

In Lustre, there is a way to declare that a set of processes all share a single write lock over the whole file. Processes are then responsible for ensuring that they don't step on each other when writing. One has to use close/sync to push data. One also has to write data on page boundaries. More details can be found here:

https://jira.whamcloud.com/browse/LU-15391

I have a simple test case that uses this API to test performance, though I need to update it based on the discussion in the noted ticket.

It is worth investigating how much performance one can get with this Lustre API call. If significant, then Lustre might also benefit from MPIX_File_fence. For example, when using collective I/O, the MPI library could acquire a group lock and gather data to be written on page boundaries on behalf of the user via collective buffering.

adammoody avatar Dec 23 '21 22:12 adammoody