influxdb icon indicating copy to clipboard operation
influxdb copied to clipboard

feat: Update Stream to accept a bufSize for tar CopyBuffer window

Open devanbenz opened this issue 1 month ago • 0 comments

io.Copy & io.CopyN uses a default window size of 32KB during copy

Copy(N) calls copyBuffer with buf set to nil https://cs.opensource.google/go/go/+/refs/tags/go1.25.4:src/io/io.go;l=387-389

When buf is nil go will set the default buffer size to 32KB https://cs.opensource.google/go/go/+/refs/tags/go1.25.4:src/io/io.go;l=418

This PR replaces io.CopyN with io.CopyBuffer in tar.Stream. It also adds a new configuration value tar-stream-buffer-size which can be adjusted to allow for larger or smaller tar buffer sizes to be used during backup operations.

It is suggested to keep the default as is, while testing different buffer sizes to be used with CopyBuffer I found that 1MB was the best performance-wise.

devanbenz avatar Nov 06 '25 21:11 devanbenz