ngo icon indicating copy to clipboard operation
ngo copied to clipboard

Improve the scalability of SyncIoDisk

Open tatetian opened this issue 2 years ago • 4 comments

The SyncIoDisk in sgx-disk is not scalable on concurrency.

image

The problem is that there is a lock hold while doing I/O. This lock can be removed if using read_at and write_at API.

image

tatetian avatar Jun 06 '22 00:06 tatetian

Workaround: Use std::os::unix::fs::FileExt::read_at and write_at API, remove the lock. Bench result: image Code: image

lucassong-mh avatar Jun 07 '22 02:06 lucassong-mh

This is good perf improvement. But the implementation contains a bug... The offset should be calculated as follows.

offset += buf.as_slice().len()

Another thing is that the implementation should better use pwritev so that the number of OCalls is always one per request, instead of one per buffer.

tatetian avatar Jun 07 '22 03:06 tatetian

Improvement: Use libc::ocall::preadv64andpwritev64 API. Bench result: image Code: image

lucassong-mh avatar Jun 08 '22 02:06 lucassong-mh

Great. Now we can submit a pull request.

tatetian avatar Jun 09 '22 01:06 tatetian