rqbit icon indicating copy to clipboard operation
rqbit copied to clipboard

use pwritev and preadv on linux

Open milahu opened this issue 1 year ago • 1 comments

https://docs.rs/tokio-file/latest/tokio_file/trait.AioFileExt.html#method.writev_at

https://docs.rs/tokio-file/latest/tokio_file/trait.AioFileExt.html#method.readv_at

https://github.com/rust-lang/rust/issues/71055

https://github.com/vimpunk/cratetorrent/blob/master/DESIGN.md#vectored-io

https://old.reddit.com/r/rust/comments/kiah3q/i_wrote_cratetorrent_a_bittorrent_engine_in_rust/

https://github.com/vimpunk/cratetorrent/pull/95

milahu avatar Jun 20 '24 14:06 milahu

Blocks are not buffered in memory but dumped straight to disk. Once all are written, the data is validated. This has a nice property of not taking any memory. In fact, the only extra memory taken is the peer read buffer which is static per peer.

This was working well so far and is simple enough. However recently I added a storage trait where you can do whatever else - e.g. buffer (cache) blocks and write them with pwritev to reduce system calls.

That extra complexity and extra memory taken though should be justified - and I haven't seen yet why it would.

ikatson avatar Jun 20 '24 15:06 ikatson

Won't do this, so closing. The current design of not buffering anything is perfectly fine, and may be even superior to the buffering approach which would allow for pwritev/preadv. Not that long ago pwrite and pread were added instead to allow lockless concurrent reads/writes to files, which should be good enough.

ikatson avatar Aug 27 '24 21:08 ikatson