pv icon indicating copy to clipboard operation
pv copied to clipboard

Don't cap read/write buffer size?

Open nh2 opened this issue 1 year ago • 0 comments

I tried to tell pv to make large read()/write() sycalls (it is much more efficient to make large syscalls on e.g. networked file systems) using --buffer-size 16M.

However, I found that it had no effect; pv was making 512Ki-sized IO operations.

This is due to:

#define MAX_READ_AT_ONCE	524288	 /* max to read() in one go */
#define MAX_WRITE_AT_ONCE	524288	 /* max to write() in one go */

// ...

		    read(fd, buf,
			 count >
			 MAX_READ_AT_ONCE ? MAX_READ_AT_ONCE : count);

Why do these limits exisst?

Is there a point to allowing a --buffer-size 16M when the reads and writes are capped to small buffers anyway?

Could this restriction be lifted, removing the hardcoded limits, so that these syscalls can be as large as the --buffer size?

Thanks

nh2 avatar Jun 19 '23 14:06 nh2