golang
os: NewFile blocks if handle is blocked
Go version
go version go1.25.1 windows/arm64
Output of go env in your module/workspace:
set AR=ar
set CC=gcc
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_ENABLED=1
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set CXX=g++
set GCCGO=gccgo
set GO111MODULE=
set GOARCH=arm64
set GOARM64=v8.0
set GOAUTH=netrc
set GOBIN=
set GOCACHE=C:\Users\qmuntaldiaz\AppData\Local\go-build
set GOCACHEPROG=
set GODEBUG=
set GOENV=C:\Users\qmuntaldiaz\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFIPS140=off
set GOFLAGS=
set GOGCCFLAGS=-mthreads -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=C:\Users\QMUNTA~1\AppData\Local\Temp\go-build1302331125=/tmp/go-build -gno-record-gcc-switches
set GOHOSTARCH=arm64
set GOHOSTOS=windows
set GOINSECURE=
set GOMOD=NUL
set GOMODCACHE=C:\Users\qmuntaldiaz\go\pkg\mod
set GONOPROXY=github.com/microsoft
set GONOSUMDB=github.com/microsoft
set GOOS=windows
set GOPATH=C:\Users\qmuntaldiaz\go
set GOPRIVATE=github.com/microsoft
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Users\qmuntaldiaz\sdk\go1.25.1
set GOSUMDB=sum.golang.org
set GOTELEMETRY=on
set GOTELEMETRYDIR=C:\Users\qmuntaldiaz\AppData\Roaming\go\telemetry
set GOTMPDIR=
set GOTOOLCHAIN=auto
set GOTOOLDIR=C:\Users\qmuntaldiaz\sdk\go1.25.1\pkg\tool\windows_arm64
set GOVCS=
set GOVERSION=go1.25.1
set GOWORK=
set PKG_CONFIG=pkg-config
What did you do?
Call os.NewFile using a handle that is blocked doing I/O, like in here:
package main
import (
"fmt"
"os"
"syscall"
"time"
"golang.org/x/sys/windows"
)
func main() {
const name = `\\.\pipe\go_testpipe`
h, err := windows.CreateNamedPipe(syscall.StringToUTF16Ptr(name), windows.PIPE_ACCESS_DUPLEX, 0, 1, 4096, 4096, 0, nil)
if err != nil {
panic(err)
}
pipe := os.NewFile(uintptr(h), name) // OK
defer pipe.Close()
file, err := os.Open(name)
if err != nil {
panic(err)
}
defer file.Close()
go func() {
// Give some time for the Read to start and block.
time.Sleep(100 * time.Millisecond)
fmt.Println("NewFile()")
os.NewFile(uintptr(file.Fd()), name) // Unexpected block
fmt.Println("NewFile() returned")
}()
fmt.Println("Read()")
var tmp [1]byte
// Read will blocks
file.Read(tmp[:]) // Expected block
fmt.Println("Read() returned")
}
What did you see happen?
os.NewFile blocks until the handle is unblocked:
Read()
NewFile()
What did you expect to see?
os.NewFile doesn't block even if the handle is blocked:
Read()
NewFile()
NewFile() returned"
Note that this has always been the behavior until Go 1.25. The regression was introduced in CL 662236.
This situation is more common than one would expect because the os package calls NewFile(uintptr(syscall.Stdin), "/dev/stdin") when initializing. And if someone has passed a blocked stdin handle, then the os package initialization hangs. This means that a Go application can hand even before calling any user-controlled code. See for example #75949.
@golang/windows
@gopherbot please backport to Go 1.25. This is a regression introduced in Go 1.25 that can cause hangs while initializing the standard library.
Backport issue(s) opened: #76392 (for 1.25).
Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.
Change https://go.dev/cl/718002 mentions this issue: os,internal/poll: lazily add a handle to IOCP on Windows
Related Issues
- os: NewFile blocks if handle is blocked [1.25 backport] #76392
- os: user-provided message named pipes can't be closed #73084 (closed)
- os: OpenFile using O_TRUNC and O_CREATE on a named pipe fails on windows #76071 (closed)
- os: File.Seek doesn't set the correct offset with Windows overlapped handles #75081 (closed)
(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)
Change https://go.dev/cl/724640 mentions this issue: os,internal/poll: don't call IsNonblock for consoles and Stdin
@qmuntal The commit message of CL 724640 contained "This avoids potential deadlocks during os package initialization" and "Updates #76391", so this issue is still open. Is there more needed to resolve this for Go 1.26 (and to be backported)?
CL 724640 doesn't completely fix the hand in NewFile, it makes it less likely and also avoids it to occur when initializing the os package. So yes, it should be backported (will do that myself).
This issue should remain open until os.NewFile can't block, but the fix (if any) might be to complex to be backported.
Change https://go.dev/cl/725580 mentions this issue: [release-branch.go1.25] os,internal/poll: don't call IsNonblock for consoles and Stdin
Thanks for clarifying. Given the remaining fix might be complex, do you think it's worth trying to get it into Go 1.26, while we're still fairly early in the freeze? I applied the release-blocker label since this seemed to need attention, but maybe the incomplete fix of CL 724640 is enough to resolve that, and the rest of this can be left for Go 1.27 or later.
The release timeline targets next week for Go 1.26 RC 1 and this issue currently blocks it. If you think it's safe enough to target RC 2 (tentatively early January), we can do that instead. Or, leave the rest for Go 1.27+ as mentioned above, if that's more appropriate. Thanks.
We are facing a similar issue at Meta with hung Windows binaries, which is preventing us from deprecating Go 1.24.
I was able to obtain a stack trace using dlv attach, which led us to the os.NewFile function.
Full stack trace
0 0x00007ffcfad11024 in ???
at ?:-1
1 0x00000001400774e8 in runtime.systemstack_switch
at $GOROOT/src/runtime/asm_amd64.s:478
2 0x0000000140071512 in runtime.cgocall
at $GOROOT/src/runtime/cgocall.go:185
3 0x000000014005f489 in runtime.syscall_syscalln
at $GOROOT/src/runtime/syscall_windows.go:521
4 0x0000000140075998 in syscall.Syscall6
at $GOROOT/src/runtime/syscall_windows.go:463
5 0x00000001400b144f in internal/syscall/windows.NtQueryInformationFile
at $GOROOT/src/internal/syscall/windows/zsyscall_windows.go:537
6 0x00000001400b0b2f in internal/syscall/windows.IsNonblock
at $GOROOT/src/internal/syscall/windows/nonblocking_windows.go:17
7 0x00000001400b792c in os.newFileFromNewFile
at $GOROOT/src/os/file_windows.go:90
8 0x00000001400b6d9a in os.NewFile
at $GOROOT/src/os/file.go:134
9 0x00000001400b5a5a in os.init
at <autogenerated>:1
10 0x0000000140053114 in runtime.doInit1
at $GOROOT/src/runtime/proc.go:7670
11 0x000000014007396a in runtime.doInit
at $GOROOT/src/runtime/proc.go:7637
12 0x000000014004525e in runtime.main
at $GOROOT/src/runtime/proc.go:256
13 0x0000000140079501 in runtime.goexit
at $GOROOT/src/runtime/asm_amd64.s:1693
The good news is that I can confirm https://go.dev/cl/725580 resolves the issue. Thank you for working on it! 😀
Thanks for clarifying. Given the remaining fix might be complex, do you think it's worth trying to get it into Go 1.26, while we're still fairly early in the freeze?
I will give it another try for RC2 👍.
