fasthttp icon indicating copy to clipboard operation
fasthttp copied to clipboard

copyZeroAlloc allocates when interface uses genericWriteTo

Open lunixbochs opened this issue 1 year ago • 1 comments

it looks like go's io.CopyBuffer always uses the WriterTo interface if it's available, much like copyZeroAlloc

however, File and TCPConn unfortunately have a fallback in their WriterTo that calls io.Copy if sendfile isn't possible, which allocates an implicit buffer

https://github.com/golang/go/blob/15c558016088d6aaf103b4f0fd2b716a4573e5a2/src/net/net.go#L789 https://github.com/golang/go/blob/15c558016088d6aaf103b4f0fd2b716a4573e5a2/src/os/file.go#L274

this is with an os.File reaching writeBodyFixedSize

lunixbochs avatar Oct 25 '24 02:10 lunixbochs

it doesn't look like the golang zero copy backend works with bufio.Writer at all, despite the comment here https://github.com/valyala/fasthttp/blob/40bdc4abc74e2760e4c3b3153b2d4c2357eb7f4e/http.go#L2205-L2211

the os.File writeTo implementation will see that the destination is not a network socket and immediately fall back to io.Copy, which allocates an implicit buffer: https://github.com/golang/go/blob/15c558016088d6aaf103b4f0fd2b716a4573e5a2/src/os/zero_copy_linux.go#L19

func (f *File) writeTo(w io.Writer) (written int64, handled bool, err error) {
	pfd, network := getPollFDAndNetwork(w)
	// TODO(panjf2000): same as File.spliceToFile.
	if pfd == nil || !pfd.IsStream || !isUnixOrTCP(string(network)) {
		return
	}

https://github.com/golang/go/blob/15c558016088d6aaf103b4f0fd2b716a4573e5a2/src/os/zero_copy_linux.go#L163

func getPollFDAndNetwork(i any) (*poll.FD, poll.String) {
	sc, ok := i.(syscall.Conn)
	if !ok {
		return nil, ""
	}

lunixbochs avatar Oct 25 '24 03:10 lunixbochs

@erikdubbelboer This was fixed in https://github.com/valyala/fasthttp/pull/1893 it can be closed

gaby avatar Nov 11 '25 00:11 gaby