go
go copied to clipboard
runtime: TestTSAN: fails with `signal handler spoils errno`
#!watchflakes
post <- pkg == "cmd/cgo/internal/testsanitizers" && test ~ `TestTSAN` && `signal handler spoils errno`
Go version
go version go1.22.1 linux/amd64 (go 1.21.7 for bootstrap stage)
Output of go env
in your module/workspace:
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/tim/.cache/go-build'
GOENV='/home/tim/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/tim/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/tim/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/tim/Repositories/goroot'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/tim/Repositories/goroot/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.1'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/tim/Repositories/goroot/src/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3241501588=/tmp/go-build -gno-record-gcc-switches'
What did you do?
git checkout go1.22.1
cd src
./all.bash
What did you see happen?
The following test fails sometimes (30-50% of the builds). Usually retrying the all.bash
script works. I haven't noticed a similar error on the 1.21 tags, but that might have been pure luck, or something was introduced in 1.22 that causes this flakiness.
--- FAIL: TestTSAN (0.92s)
--- FAIL: TestTSAN/tsan14 (3.64s)
tsan_test.go:77: /tmp/TestTSAN756015491/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=13873)
Signal 28 handler invoked at:
#0 runtime.cgoSigtramp /home/tim/Repositories/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x5164c0) (BuildId: ee54abde780f2dee550d853b6a654a5328fde482)
#1 _cgo_release_context <null> (tsan14+0x518d90) (BuildId: ee54abde780f2dee550d853b6a654a5328fde482)
#2 go_callback <null> (tsan14+0x518ac3) (BuildId: ee54abde780f2dee550d853b6a654a5328fde482)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/tim/Repositories/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
FAIL
FAIL cmd/cgo/internal/testsanitizers 6.244s
What did you expect to see?
Test to pass each time.
I tried looking if there is already a similar issue, but most issues have to do with sigfaults or other panics. I did not find this specific warning.
Because I fear this might end up being relevant (seems like we'd notice inability to run all.bash on Linux, it happens constantly) could you also supply Linux and Gcc version information? No linker preload or nonstandard path, that sort of thing?
@golang/runtime
OS release info:
NAME="openSUSE Tumbleweed"
# VERSION="20240302"
ID="opensuse-tumbleweed"
ID_LIKE="opensuse suse"
VERSION_ID="20240302"
PRETTY_NAME="openSUSE Tumbleweed"
ANSI_COLOR="0;32"
# CPE 2.3 format, boo#1217921
CPE_NAME="cpe:2.3:o:opensuse:tumbleweed:20240302:*:*:*:*:*:*:*"
#CPE 2.2 format
#CPE_NAME="cpe:/o:opensuse:tumbleweed:20240302"
BUG_REPORT_URL="https://bugzilla.opensuse.org"
SUPPORT_URL="https://bugs.opensuse.org"
HOME_URL="https://www.opensuse.org"
DOCUMENTATION_URL="https://en.opensuse.org/Portal:Tumbleweed"
LOGO="distributor-logo-Tumbleweed"
GCC info:
`gcc -dumpspec` output
gcc -dumpspecs
*asm:
%{m16|m32:--32} %{m16|m32|mx32:;:--64} %{mx32:--x32} %{msse2avx:%{!mavx:-msse2avx}}
*asm_debug:
%{g*:%{%:debug-level-gt(0):}} %{ffile-prefix-map=*:--debug-prefix-map %*} %{fdebug-prefix-map=*:--debug-prefix-map %*}
*asm_debug_option:
%{g*:%{%:debug-level-gt(0):%{%:dwarf-version-gt(4):--gdwarf-5 ;%:dwarf-version-gt(3):--gdwarf-4 ;%:dwarf-version-gt(2):--gdwarf-3 ;:--gdwarf2 }}}
*asm_final:
%{gsplit-dwarf:
objcopy --extract-dwo %{c:%{o*:%*}%{!o*:%w%b%O}}%{!c:%U%O} %b.dwo
objcopy --strip-dwo %{c:%{o*:%*}%{!o*:%w%b%O}}%{!c:%U%O} }
*asm_options:
%{-target-help:%:print-asm-header()} %{v} %{w:-W} %{I*} %(asm_debug_option) %{gz|gz=zlib:--compress-debug-sections=zlib} %{gz=none:--compress-debug-sections=none} %{gz=zstd:--compress-debug-sections=zstd} %{gz=zlib-gnu:}%a %Y %{c:%W{o*}%{!o*:-o %w%b%O}}%{!c:-o %d%w%u%O}
*invoke_as:
%{!fwpa*: %{fcompare-debug=*|fdump-final-insns=*:%:compare-debug-dump-opt()} %{!S:-o %|.s |
as %(asm_options) %m.s %A } }
*cpp:
%{posix:-D_POSIX_SOURCE} %{pthread:-D_REENTRANT}
*cpp_options:
%(cpp_unique_options) %1 %{m*} %{std*&ansi&trigraphs} %{W*&pedantic*} %{w} %{f*} %{g*:%{%:debug-level-gt(0):%{g*} %{!fno-working-directory:-fworking-directory}}} %{O*} %{undef} %{save-temps*:-fpch-preprocess}
*cpp_debug_options:
%<dumpdir %<dumpbase %<dumpbase-ext %{d*} %:dumps()
*cpp_unique_options:
%{!Q:-quiet} %{nostdinc*} %{C} %{CC} %{v} %@{I*&F*} %{P} %I %{MD:-MD %{!o:%b.d}%{o*:%.d%*}} %{MMD:-MMD %{!o:%b.d}%{o*:%.d%*}} %{M} %{MM} %{MF*} %{MG} %{MP} %{MQ*} %{MT*} %{Mmodules} %{Mno-modules} %{!E:%{!M:%{!MM:%{!MT:%{!MQ:%{MD|MMD:%{o*:-MQ %*}}}}}}} %{remap} %{%:debug-level-gt(2):-dD} %{!iplugindir*:%{fplugin*:%:find-plugindir()}} %{H} %C %{D*&U*&A*} %{i*} %Z %i %{E|M|MM:%W{o*}}
*trad_capable_cpp:
cc1 -E %{traditional|traditional-cpp:-traditional-cpp}
*cc1:
%{!mandroid|tno-android-cc:%(cc1_cpu) %{profile:-p};:%(cc1_cpu) %{profile:-p} %{!mglibc:%{!muclibc:%{!mbionic: -mbionic}}} %{!fno-pic:%{!fno-PIC:%{!fpic:%{!fPIC: -fPIC}}}}}
*cc1_options:
%{pg:%{fomit-frame-pointer:%e-pg and -fomit-frame-pointer are incompatible}} %{!iplugindir*:%{fplugin*:%:find-plugindir()}} %1 %{!Q:-quiet} %(cpp_debug_options) %{m*} %{aux-info*} %{g*} %{O*} %{W*&pedantic*} %{w} %{std*&ansi&trigraphs} %{v:-version} %{pg:-p} %{p} %{f*} %{undef} %{Qn:-fno-ident} %{Qy:} %{-help:--help} %{-target-help:--target-help} %{-version:--version} %{-help=*:--help=%*} %{!fsyntax-only:%{S:%W{o*}%{!o*:-o %w%b.s}}} %{fsyntax-only:-o %j} %{-param*} %{coverage:-fprofile-arcs -ftest-coverage} %{fprofile-arcs|fprofile-generate*|coverage: %{!fprofile-update=single: %{pthread:-fprofile-update=prefer-atomic}}}
*cc1plus:
*link_gcc_c_sequence:
%{static|static-pie:--start-group} %G %{!nolibc:%L} %{static|static-pie:--end-group}%{!static:%{!static-pie:%G}}
*link_ssp:
%{fstack-protector|fstack-protector-all|fstack-protector-strong|fstack-protector-explicit:}
*endfile:
%{!mandroid|tno-android-ld:%{mdaz-ftz:crtfastmath.o%s;Ofast|ffast-math|funsafe-math-optimizations:%{!shared:%{!mno-daz-ftz:crtfastmath.o%s}}} %{mpc32:crtprec32.o%s} %{mpc64:crtprec64.o%s} %{mpc80:crtprec80.o%s} %{!static:%{fvtable-verify=none:%s; fvtable-verify=preinit:vtv_end_preinit.o%s; fvtable-verify=std:vtv_end.o%s}} %{static:crtend.o%s; shared|static-pie|pie:crtendS.o%s; :crtend.o%s} crtn.o%s %{fopenacc|fopenmp:crtoffloadend%O%s};:%{mdaz-ftz:crtfastmath.o%s;Ofast|ffast-math|funsafe-math-optimizations:%{!shared:%{!mno-daz-ftz:crtfastmath.o%s}}} %{mpc32:crtprec32.o%s} %{mpc64:crtprec64.o%s} %{mpc80:crtprec80.o%s} %{shared: crtend_so%O%s;: crtend_android%O%s}}
*link:
%{!r:--build-id} %{!static|static-pie:--eh-frame-hdr} %{!mandroid|tno-android-ld:%{m16|m32|mx32:;:-m elf_x86_64} %{m16|m32:-m elf_i386} %{mx32:-m elf32_x86_64} %{shared:-shared} %{!shared: %{!static: %{!static-pie: %{rdynamic:-export-dynamic} %{m16|m32:-dynamic-linker %{muclibc:/lib/ld-uClibc.so.0;:%{mbionic:/system/bin/linker;:%{mmusl:/lib/ld-musl-i386.so.1;:/lib/ld-linux.so.2}}}} %{m16|m32|mx32:;:-dynamic-linker %{muclibc:/lib/ld64-uClibc.so.0;:%{mbionic:/system/bin/linker64;:%{mmusl:/lib/ld-musl-x86_64.so.1;:/lib64/ld-linux-x86-64.so.2}}}} %{mx32:-dynamic-linker %{muclibc:/lib/ldx32-uClibc.so.0;:%{mbionic:/system/bin/linkerx32;:%{mmusl:/lib/ld-musl-x32.so.1;:/libx32/ld-linux-x32.so.2}}}}}} %{static:-static} %{static-pie:-static -pie --no-dynamic-linker -z text}};:%{m16|m32|mx32:;:-m elf_x86_64} %{m16|m32:-m elf_i386} %{mx32:-m elf32_x86_64} %{shared:-shared} %{!shared: %{!static: %{!static-pie: %{rdynamic:-export-dynamic} %{m16|m32:-dynamic-linker %{muclibc:/lib/ld-uClibc.so.0;:%{mbionic:/system/bin/linker;:%{mmusl:/lib/ld-musl-i386.so.1;:/lib/ld-linux.so.2}}}} %{m16|m32|mx32:;:-dynamic-linker %{muclibc:/lib/ld64-uClibc.so.0;:%{mbionic:/system/bin/linker64;:%{mmusl:/lib/ld-musl-x86_64.so.1;:/lib64/ld-linux-x86-64.so.2}}}} %{mx32:-dynamic-linker %{muclibc:/lib/ldx32-uClibc.so.0;:%{mbionic:/system/bin/linkerx32;:%{mmusl:/lib/ld-musl-x32.so.1;:/libx32/ld-linux-x32.so.2}}}}}} %{static:-static} %{static-pie:-static -pie --no-dynamic-linker -z text}} %{shared: -Bsymbolic}}
*lib:
%{!mandroid|tno-android-ld:%{pthread:-lpthread} %{shared:-lc} %{!shared:%{profile:-lc_p}%{!profile:-lc}};:%{shared:-lc} %{!shared:%{profile:-lc_p}%{!profile:-lc}} %{!static: -ldl}}
*link_gomp:
*libgcc:
%{static|static-libgcc|static-pie:-lgcc -lgcc_eh}%{!static:%{!static-libgcc:%{!static-pie:%{!shared-libgcc:-lgcc --push-state --as-needed -lgcc_s --pop-state}%{shared-libgcc:-lgcc_s%{!shared: -lgcc}}}}}
*startfile:
%{!mandroid|tno-android-ld:%{shared:; pg|p|profile:%{static-pie:grcrt1.o%s;:gcrt1.o%s}; static:crt1.o%s; static-pie:rcrt1.o%s; pie:Scrt1.o%s; :crt1.o%s} crti.o%s %{static:crtbeginT.o%s; shared|static-pie|pie:crtbeginS.o%s; :crtbegin.o%s} %{fvtable-verify=none:%s; fvtable-verify=preinit:vtv_start_preinit.o%s; fvtable-verify=std:vtv_start.o%s} %{fopenacc|fopenmp:crtoffloadbegin%O%s};:%{shared: crtbegin_so%O%s;: %{static: crtbegin_static%O%s;: crtbegin_dynamic%O%s}}}
*cross_compile:
0
*version:
13.2.1
*multilib:
. !m64 !m32;64:../lib64 m64 !m32;32:../lib !m64 m32;
*multilib_defaults:
m64
*multilib_extra:
*multilib_matches:
m64 m64;m32 m32;
*multilib_exclusions:
*multilib_options:
m64/m32
*multilib_reuse:
*linker:
collect2
*linker_plugin_file:
*lto_wrapper:
*lto_gcc:
*post_link:
*link_libgcc:
%D
*md_exec_prefix:
*md_startfile_prefix:
*md_startfile_prefix_1:
*startfile_prefix_spec:
*sysroot_spec:
--sysroot=%R
*sysroot_suffix_spec:
*sysroot_hdrs_suffix_spec:
*self_spec:
*cc1_cpu:
%{march=native:%>march=native %:local_cpu_detect(arch %{!m32:64;:32}) %{!mtune=*:%>mtune=native %:local_cpu_detect(tune %{!m32:64;:32})}} %{mtune=native:%>mtune=native %:local_cpu_detect(tune %{!m32:64;:32})}
*link_command:
%{!fsyntax-only:%{!c:%{!M:%{!MM:%{!E:%{!S: %(linker) %{!fno-use-linker-plugin:%{!fno-lto: -plugin %(linker_plugin_file) -plugin-opt=%(lto_wrapper) -plugin-opt=-fresolution=%u.res %{flinker-output=*:-plugin-opt=-linker-output-known} %{!nostdlib:%{!nodefaultlibs:%:pass-through-libs(%(link_gcc_c_sequence))}} }}%{flto|flto=*:%<fcompare-debug*} %{flto} %{fno-lto} %{flto=*} %l %{static|shared|r:;pie:-pie} %{fuse-ld=*:-fuse-ld=%*} %{gz|gz=zlib:--compress-debug-sections=zlib} %{gz=none:--compress-debug-sections=none} %{gz=zstd:--compress-debug-sections=zstd} %{gz=zlib-gnu:}%X %{o*} %{e*} %{N} %{n} %{r} %{s} %{t} %{u*} %{z} %{Z} %{!nostdlib:%{!r:%{!nostartfiles:%S}}} %{static|no-pie|static-pie:} %@{L*} %(mfwrap) %(link_libgcc) %{fvtable-verify=none:} %{fvtable-verify=std: %e-fvtable-verify=std is not supported in this configuration} %{fvtable-verify=preinit: %e-fvtable-verify=preinit is not supported in this configuration} %{!nostdlib:%{!r:%{!nodefaultlibs:%{%:sanitize(address):%{!shared:libasan_preinit%O%s} %{static-libasan:%{!shared:-Bstatic --whole-archive -lasan --no-whole-archive -Bdynamic}}%{!static-libasan:-lasan}} %{%:sanitize(hwaddress):%{!shared:libhwasan_preinit%O%s} %{static-libhwasan:%{!shared:-Bstatic --whole-archive -lhwasan --no-whole-archive -Bdynamic}}%{!static-libhwasan:-lhwasan}} %{%:sanitize(thread):%{!shared:libtsan_preinit%O%s} %{static-libtsan:%{!shared:-Bstatic --whole-archive -ltsan --no-whole-archive -Bdynamic}}%{!static-libtsan:-ltsan}} %{%:sanitize(leak):%{!shared:liblsan_preinit%O%s} %{static-liblsan:%{!shared:-Bstatic --whole-archive -llsan --no-whole-archive -Bdynamic}}%{!static-liblsan:-llsan}}}}} %o %{fopenacc|fopenmp|%:gt(%{ftree-parallelize-loops=*:%*} 1): %:include(libgomp.spec)%(link_gomp)} %{fgnu-tm:%:include(libitm.spec)%(link_itm)} %(mflib) %{fsplit-stack: --wrap=pthread_create} %{fprofile-arcs|fprofile-generate*|coverage:-lgcov} %{!nostdlib:%{!r:%{!nodefaultlibs:%{%:sanitize(address): %{static-libasan|static:%:include(libsanitizer.spec)%(link_libasan)} %{static:%ecannot specify -static with -fsanitize=address}} %{%:sanitize(hwaddress): %{static-libhwasan|static:%:include(libsanitizer.spec)%(link_libhwasan)} %{static:%ecannot specify -static with -fsanitize=hwaddress}} %{%:sanitize(thread): %{static-libtsan|static:%:include(libsanitizer.spec)%(link_libtsan)} %{static:%ecannot specify -static with -fsanitize=thread}} %{%:sanitize(undefined):%{static-libubsan:-Bstatic} -lubsan %{static-libubsan:-Bdynamic} %{static-libubsan|static:%:include(libsanitizer.spec)%(link_libubsan)}} %{%:sanitize(leak): %{static-liblsan|static:%:include(libsanitizer.spec)%(link_liblsan)}}}}} %{!nostdlib:%{!r:%{!nodefaultlibs:%(link_ssp) %(link_gcc_c_sequence)}}} %{!nostdlib:%{!r:%{!nostartfiles:%E}}} %{T*}
%(post_link) }}}}}}
echo $LD_PRELOAD
return nothing. I'm not sure if OpenSUSE enables this by default, but I for sure didn't do that myself.
Path is non-standard because I need the go paths in there:
/home/tim/go/bin:/home/tim/Repositories/goroot/bin:/home/tim/.local/bin:/home/tim/bin:/usr/local/bin:/usr/bin:/bin
We are seeing this on our builders as well. For example: https://ci.chromium.org/ui/p/golang/builders/ci/gotip-linux-amd64-staticlockranking/b8752359739889548657/overview
This seems like a recent regression.
Huh, @muhlemmer reports this on 1.22.1, not tip. On our builders, it looks like a very recent regression. https://ci.chromium.org/ui/test/golang/cmd%2Fcgo%2Finternal%2Ftestsanitizers.TestTSAN%2Ftsan14 shows the first failure was on 2024-03-25 in https://ci.chromium.org/ui/p/golang/builders/try/gotip-linux-amd64-boringcrypto/b8752453947853961217/test-results?sortby=&groupby=.
So maybe there is some external factor here?
Found new dashboard test flakes for:
#!watchflakes
post <- pkg == "cmd/cgo/internal/testsanitizers" && test ~ `TestTSAN` && `signal handler spoils errno`
2024-03-26 23:38 gotip-linux-amd64-staticlockranking go@50dcffb3 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN3253651474/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=36645)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x512660)
#1 _cgo_wait_runtime_init_done <null> (tsan14+0x5198d1)
#2 go_callback <null> (tsan14+0x5192f3)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (2.87s)
2024-03-27 19:06 gotip-linux-amd64-boringcrypto go@2860e018 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN490279209/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=277640)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510380)
#1 _cgo_get_context_function <null> (tsan14+0x517922)
#2 _cgo_release_context <null> (tsan14+0x517330)
#3 go_callback <null> (tsan14+0x517063)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.73s)
2024-03-30 00:20 gotip-linux-amd64-staticlockranking go@ba9c445f cmd/cgo/internal/testsanitizers.TestTSAN/tsan13 (log)
=== RUN TestTSAN/tsan13
=== PAUSE TestTSAN/tsan13
=== CONT TestTSAN/tsan13
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN3846793104/tsan13 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=70509)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan13+0x519900)
#1 spin /home/swarming/.swarming/w/ir/x/w/goroot/src/cmd/cgo/internal/testsanitizers/testdata/tsan13.go:43 (tsan13+0x55fa1f)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan13 (3.62s)
2024-04-01 12:38 gotip-linux-amd64-newinliner go@6bfaafd3 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN4222084244/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=39316)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x5104c0)
#1 _cgo_get_context_function <null> (tsan14+0x517ae2)
#2 _cgo_release_context <null> (tsan14+0x5174f0)
#3 go_callback <null> (tsan14+0x517223)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.80s)
2024-04-01 20:28 gotip-linux-amd64 go@cd294f55 cmd/cgo/internal/testsanitizers.TestTSAN/tsan13 (log)
=== RUN TestTSAN/tsan13
=== PAUSE TestTSAN/tsan13
=== CONT TestTSAN/tsan13
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN2522148544/tsan13 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=73324)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan13+0x517580)
#1 spin /home/swarming/.swarming/w/ir/x/w/goroot/src/cmd/cgo/internal/testsanitizers/testdata/tsan13.go:43 (tsan13+0x55d69f)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan13 (4.07s)
2024-04-02 16:25 gotip-linux-amd64-newinliner go@94dba612 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN664116226/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=36900)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x5104a0)
#1 _cgo_get_context_function <null> (tsan14+0x517ac2)
#2 _cgo_release_context <null> (tsan14+0x5174d0)
#3 go_callback <null> (tsan14+0x517203)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.36s)
2024-04-03 22:44 gotip-linux-amd64-staticlockranking go@23fc9170 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN237925155/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=54209)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x512840)
#1 _cgo_wait_runtime_init_done <null> (tsan14+0x519ab1)
#2 go_callback <null> (tsan14+0x5194d3)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.93s)
2024-04-05 18:02 gotip-linux-amd64-newinliner go@62791eb4 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN1675893496/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=639609)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x5104a0)
#1 _cgo_wait_runtime_init_done <null> (tsan14+0x517791)
#2 go_callback <null> (tsan14+0x5171b3)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (5.47s)
2024-04-05 22:09 gotip-linux-amd64-staticlockranking go@d186dde8 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN2177321389/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=38790)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x512d40)
#1 _cgo_get_context_function <null> (tsan14+0x51a2e2)
#2 _cgo_release_context <null> (tsan14+0x519cf0)
#3 go_callback <null> (tsan14+0x519a23)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (2.94s)
2024-04-08 20:51 gotip-linux-amd64-boringcrypto go@8008998b cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN2396447065/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=540472)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510a60)
#1 _cgo_get_context_function <null> (tsan14+0x518002)
#2 _cgo_release_context <null> (tsan14+0x517a10)
#3 go_callback <null> (tsan14+0x517743)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.29s)
2024-04-09 14:35 gotip-linux-amd64-newinliner go@de3a3c9e cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN532251830/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=40383)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510ae0)
#1 _cgo_wait_runtime_init_done <null> (tsan14+0x517dd1)
#2 go_callback <null> (tsan14+0x5177f3)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.76s)
2024-04-10 16:37 gotip-linux-amd64-boringcrypto go@1bac2528 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN588248903/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=34867)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510b80)
#1 _cgo_wait_runtime_init_done <null> (tsan14+0x517df1)
#2 go_callback <null> (tsan14+0x517813)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.87s)
2024-04-14 18:17 gotip-linux-amd64-newinliner go@37f48222 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN2490600151/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=49679)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510b80)
#1 _cgo_wait_runtime_init_done <null> (tsan14+0x517e71)
#2 go_callback <null> (tsan14+0x517893)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.83s)
2024-04-16 20:27 gotip-linux-amd64-newinliner go@661f9814 cmd/cgo/internal/testsanitizers.TestTSAN/tsan13 (log)
=== RUN TestTSAN/tsan13
=== PAUSE TestTSAN/tsan13
=== CONT TestTSAN/tsan13
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN3313030443/tsan13 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=312788)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan13+0x517c20)
#1 spin /home/swarming/.swarming/w/ir/x/w/goroot/src/cmd/cgo/internal/testsanitizers/testdata/tsan13.go:43 (tsan13+0x55e69f)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan13 (3.28s)
2024-04-17 16:36 gotip-linux-amd64 go@f367fea8 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN124908992/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=139306)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510c20)
#1 _cgo_get_context_function <null> (tsan14+0x5181c2)
#2 _cgo_release_context <null> (tsan14+0x517bd0)
#3 go_callback <null> (tsan14+0x517903)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.20s)
2024-04-17 19:54 gotip-linux-amd64-goamd64v3 go@2073b35e cmd/cgo/internal/testsanitizers.TestTSAN/tsan13 (log)
=== RUN TestTSAN/tsan13
=== PAUSE TestTSAN/tsan13
=== CONT TestTSAN/tsan13
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN2378639952/tsan13 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=182504)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan13+0x5177a0)
#1 spin /home/swarming/.swarming/w/ir/x/w/goroot/src/cmd/cgo/internal/testsanitizers/testdata/tsan13.go:43 (tsan13+0x55d75f)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan13 (3.63s)
2024-04-17 21:11 gotip-linux-amd64-boringcrypto go@334ce510 cmd/cgo/internal/testsanitizers.TestTSAN/tsan13 (log)
=== RUN TestTSAN/tsan13
=== PAUSE TestTSAN/tsan13
=== CONT TestTSAN/tsan13
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN3929203442/tsan13 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=35007)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan13+0x517c80)
#1 spin /home/swarming/.swarming/w/ir/x/w/goroot/src/cmd/cgo/internal/testsanitizers/testdata/tsan13.go:43 (tsan13+0x55ddbf)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan13 (3.28s)
2024-04-17 21:11 gotip-linux-amd64-boringcrypto go@334ce510 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN196000240/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=34901)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510c20)
#1 _cgo_get_context_function <null> (tsan14+0x5181c2)
#2 _cgo_release_context <null> (tsan14+0x517bd0)
#3 go_callback <null> (tsan14+0x517903)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.25s)
Found new dashboard test flakes for:
#!watchflakes
post <- pkg == "cmd/cgo/internal/testsanitizers" && test ~ `TestTSAN` && `signal handler spoils errno`
2024-04-19 16:08 gotip-linux-amd64-staticlockranking go@d428a638 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN1360052067/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=36513)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x513060)
#1 _cgo_get_context_function <null> (tsan14+0x51a602)
#2 _cgo_release_context <null> (tsan14+0x51a010)
#3 go_callback <null> (tsan14+0x519d43)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.50s)
2024-04-19 16:31 gotip-linux-amd64-boringcrypto go@1a0b8637 cmd/cgo/internal/testsanitizers.TestTSAN/tsan13 (log)
=== RUN TestTSAN/tsan13
=== PAUSE TestTSAN/tsan13
=== CONT TestTSAN/tsan13
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN2432848773/tsan13 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=42886)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan13+0x517de0)
#1 spin /home/swarming/.swarming/w/ir/x/w/goroot/src/cmd/cgo/internal/testsanitizers/testdata/tsan13.go:46 (tsan13+0x55ded8)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan13 (3.33s)
2024-04-19 16:31 gotip-linux-amd64-boringcrypto go@1a0b8637 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN322841745/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=41208)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510d80)
#1 _cgo_get_context_function <null> (tsan14+0x518322)
#2 _cgo_release_context <null> (tsan14+0x517d30)
#3 go_callback <null> (tsan14+0x517a63)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.33s)
Found new dashboard test flakes for:
#!watchflakes
post <- pkg == "cmd/cgo/internal/testsanitizers" && test ~ `TestTSAN` && `signal handler spoils errno`
2024-04-22 13:29 gotip-linux-amd64-boringcrypto go@2dddc7ef cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN1167938126/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=43478)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510da0)
#1 _cgo_wait_runtime_init_done <null> (tsan14+0x518011)
#2 go_callback <null> (tsan14+0x517a33)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.06s)
Found new dashboard test flakes for:
#!watchflakes
post <- pkg == "cmd/cgo/internal/testsanitizers" && test ~ `TestTSAN` && `signal handler spoils errno`
2024-04-22 20:21 gotip-linux-amd64 go@6737f4ce cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN2314264949/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=45198)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510e20)
#1 _cgo_get_context_function <null> (tsan14+0x5183c2)
#2 _cgo_release_context <null> (tsan14+0x517dd0)
#3 go_callback <null> (tsan14+0x517b03)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.90s)
Found new dashboard test flakes for:
#!watchflakes
post <- pkg == "cmd/cgo/internal/testsanitizers" && test ~ `TestTSAN` && `signal handler spoils errno`
2024-04-23 16:49 gotip-linux-amd64-goamd64v3 go@08e73e61 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN3886674001/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=269999)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510940)
#1 _cgo_wait_runtime_init_done <null> (tsan14+0x517b51)
#2 go_callback <null> (tsan14+0x517573)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (2.77s)
2024-04-23 16:49 gotip-linux-amd64-staticlockranking go@08e73e61 cmd/cgo/internal/testsanitizers.TestTSAN/tsan13 (log)
=== RUN TestTSAN/tsan13
=== PAUSE TestTSAN/tsan13
=== CONT TestTSAN/tsan13
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN458796977/tsan13 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=204973)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan13+0x51a180)
#1 spin /home/swarming/.swarming/w/ir/x/w/goroot/src/cmd/cgo/internal/testsanitizers/testdata/tsan13.go:43 (tsan13+0x56021f)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan13 (3.58s)
TSAN implementation of errno spoiling: https://github.com/llvm/llvm-project/blob/d5224b73ccd09a6759759791f58426b6acd4a2e2/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp#L2123
Found new dashboard test flakes for:
#!watchflakes
post <- pkg == "cmd/cgo/internal/testsanitizers" && test ~ `TestTSAN` && `signal handler spoils errno`
2024-04-24 13:36 gotip-linux-amd64-staticlockranking go@62dfa431 cmd/cgo/internal/testsanitizers.TestTSAN/tsan13 (log)
=== RUN TestTSAN/tsan13
=== PAUSE TestTSAN/tsan13
=== CONT TestTSAN/tsan13
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN2907462251/tsan13 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=37132)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan13+0x51a180)
#1 spin /home/swarming/.swarming/w/ir/x/w/goroot/src/cmd/cgo/internal/testsanitizers/testdata/tsan13.go:40 (tsan13+0x5601f6)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan13 (3.51s)
2024-04-24 15:55 gotip-linux-amd64-boringcrypto go@508e7619 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN974189749/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=38580)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510e80)
#1 _cgo_get_context_function <null> (tsan14+0x5183e2)
#2 _cgo_release_context <null> (tsan14+0x517df0)
#3 go_callback <null> (tsan14+0x517b23)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.07s)
2024-04-24 15:55 gotip-linux-amd64-goamd64v3 go@508e7619 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN2446667033/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=97922)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510940)
#1 _cgo_wait_runtime_init_done <null> (tsan14+0x517b51)
#2 go_callback <null> (tsan14+0x517573)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.82s)
2024-04-25 01:02 gotip-linux-amd64-goamd64v3 go@4351af68 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN1143881792/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=286537)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x510940)
#1 _cgo_wait_runtime_init_done <null> (tsan14+0x517b51)
#2 go_callback <null> (tsan14+0x517573)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.89s)
Change https://go.dev/cl/581721 mentions this issue: runtime: debug signal handler spoiling errno
Change https://go.dev/cl/581722 mentions this issue: cmd/cgo/internal/testsanitizers: add test case for #66427
I wrote CL https://go.dev/cl/581721 to check errno at various stages in the signal handler. Here is one of the failure https://ci.chromium.org/ui/p/golang/builders/try/gotip-linux-amd64/b8749565669963801361/test-results?q=ExactID%3Acmd%2Fcgo%2Finternal%2Ftestsanitizers.TestTSAN%2Ftsan14+VHash%3A59d5ef073852df4a+&sortby=&groupby=
tsan_test.go:78: /home/swarming/.swarming/w/ir/x/t/TestTSAN4105666902/tsan14 exited with exit status 2
errno 22 sig 28 gp 0x7f78c41021c0
fatal error: errno
runtime stack:
runtime.throw({0x5403ee?, 0x0?})
/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:1021 +0x48 fp=0x7f790febec20 sp=0x7f790febebf0 pc=0x4dd7e8
runtime.checkErrno(0x1c, 0x7f78c41021c0)
/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/signal_unix.go:430 +0xca fp=0x7f790febec48 sp=0x7f790febec20 pc=0x4f2d2a
runtime.badsignal(0x1c, 0x7f790febecc0, 0x7f78c41021c0)
/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/signal_unix.go:1127 +0xa7 fp=0x7f790febec68 sp=0x7f790febec48 pc=0x4f4747
runtime.sigtrampgo(0x1c, 0x7f791001b640, 0x7f791001b6c0)
/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/signal_unix.go:486 +0x26e fp=0x7f790febece0 sp=0x7f790febec68 pc=0x4f2fae
runtime.sigtramp()
/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:361 +0x46 fp=0x7f790febed30 sp=0x7f790febece0 pc=0x510ac6
The errno is changed in the code path that we got a signal on a non-Go thread when it is not executing Go code, and we don't do anything in our signal handler but re-raise the signal. Specifically, errno changed after re-raising the signal at https://cs.opensource.google/go/go/+/master:src/runtime/signal_unix.go;l=1096 .
The signal is 28, a.k.a. SIGWINCH. Where does SIGWINCH come from? I don't know. Our test doesn't send SIGWINCH. Maybe the kernel or the builder recently starts to send SIGWINCH? And that's what changed recently? It also explains why I have not been able to reproduce locally or even on gomote (where I run the tests in a shell under ssh).
The fact that it is failing on a non-Go thread also explains why it fails in tsan13 and tsan14, but not others. Only these two tests create threads in C and call back into Go.
Based on that, I wrote another test, in CL https://go.dev/cl/581722, where it creates a thread in C, calls into Go in a loop, and at same time another thread repeatedly sends SIGWINCH to it. This fails consistently on my machine, as well as on the builders (even non-amd64).
Now that I can reproduce it, I'll keep looking into the fix, and how re-raising the signal could clobber errno... Errno 22 is EINVAL. Potentially re-raising the signal causes some invalid operation, maybe in TSAN's signal handler?
Change https://go.dev/cl/582077 mentions this issue: runtime: don't re-raise ignored signal
More debugging indicates that errno changes when we uninstall our signal handler before re-raising the signal https://cs.opensource.google/go/go/+/master:src/runtime/signal_unix.go;l=971 . This will call the TSAN-intercepted sigaction
. I'm not sure why it would change errno to 22...
At least for the case of SIGWINCH, there is no C signal handler installed, and the signal is by default ignored. In this case re-raising the signal doesn't seem to do anything. Sent CL https://go.dev/cl/582077 to not re-raise signals that are ignored. With this, errno will not change. Not sure if this is the right/best fix. Or maybe the best fix is to snapshot and restore errno around re-raising the signal?
Not sure if there is a (non-default) C signal handler installed (before Go signal handler). I think that can only happen if Go is a library (c-archive or c-shared)? In this case we only install handler for synchronous signals. And we also forward synchronous signals if it lands on a non-Go thread and there is a C handler. Maybe that is fine and we won't get to raisebadsignal
?
Found new dashboard test flakes for:
#!watchflakes
post <- pkg == "cmd/cgo/internal/testsanitizers" && test ~ `TestTSAN` && `signal handler spoils errno`
2024-04-26 21:24 gotip-linux-amd64-staticlockranking go@0e7f5cf3 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN2489155201/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=37051)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x5129c0)
#1 _cgo_get_context_function <null> (tsan14+0x519ec2)
#2 _cgo_release_context <null> (tsan14+0x5198d0)
#3 go_callback <null> (tsan14+0x519603)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.68s)
2024-04-26 23:07 gotip-linux-amd64-newinliner go@ad22356e cmd/cgo/internal/testsanitizers.TestTSAN/tsan13 (log)
=== RUN TestTSAN/tsan13
=== PAUSE TestTSAN/tsan13
=== CONT TestTSAN/tsan13
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN2526108170/tsan13 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=184056)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan13+0x517800)
#1 spin /home/swarming/.swarming/w/ir/x/w/goroot/src/cmd/cgo/internal/testsanitizers/testdata/tsan13.go:42 (tsan13+0x55e16f)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan13 (4.11s)
2024-04-26 23:07 gotip-linux-amd64-newinliner go@ad22356e cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN1452582374/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=183093)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x5107a0)
#1 _cgo_wait_runtime_init_done <null> (tsan14+0x517a11)
#2 go_callback <null> (tsan14+0x517433)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (4.10s)
Found new dashboard test flakes for:
#!watchflakes
post <- pkg == "cmd/cgo/internal/testsanitizers" && test ~ `TestTSAN` && `signal handler spoils errno`
2024-04-29 14:01 gotip-linux-amd64-newinliner go@f6e6b637 cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN1529121168/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=88729)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x5107a0)
#1 _cgo_wait_runtime_init_done <null> (tsan14+0x517a11)
#2 go_callback <null> (tsan14+0x517433)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.68s)
@cherrymui Perhaps you can run the test under strace
to see if the EINVAL is coming from a system call?
Re: SIGWINCH
, the only case I'm aware of using SIGWINCH
are window managers when a window changes size. But do these systems even have a window manager?
I tried strace
and didn't see anything helpful. I wonder that it may be the TSAN interceptor that changes errno to EINVAL before actually making a syscall? Although by reading the TSAN source code I haven't figured out where/whether it does that.
Re: SIGWINCH, the only case I'm aware of using SIGWINCH are window managers when a window changes size. But do these systems even have a window manager?
Yeah, that's pretty weird. Maybe something in the builder changed and it now thinks it has a window manager?...
Re: SIGWINCH, the only case I'm aware of using SIGWINCH are window managers when a window changes size. But do these systems even have a window manager?
That might be the trigger for the original issue I posted. I have a drop-down terminal (yakuake) I run most of my shell stuff in, like the all.bash
script. It might send SIGWINCH
every-time it collapses / expands? And if done on exactly the wrong moment it might fail the test.
I do remember that particular day I had a bunch of shell stuff to do and I multi-tasked in multiple tabs, while toggling the drop-down terminal on a frequent basis while all.bash
was running.
How it applies to pipeline tests, I have no idea.
Yeah, that's pretty weird. Maybe something in the builder changed and it now thinks it has a window manager?...
Do the builders use some kind of terminal-to-web emulator that might be sending such signals?
Found new dashboard test flakes for:
#!watchflakes
post <- pkg == "cmd/cgo/internal/testsanitizers" && test ~ `TestTSAN` && `signal handler spoils errno`
2024-05-04 07:50 gotip-linux-amd64 go@8841f50d cmd/cgo/internal/testsanitizers.TestTSAN/tsan14 (log)
=== RUN TestTSAN/tsan14
=== PAUSE TestTSAN/tsan14
=== CONT TestTSAN/tsan14
tsan_test.go:77: /home/swarming/.swarming/w/ir/x/t/TestTSAN618518940/tsan14 exited with exit status 66
==================
WARNING: ThreadSanitizer: signal handler spoils errno (pid=38752)
#0 runtime.cgoSigtramp /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 (tsan14+0x5107c0)
#1 _cgo_get_context_function <null> (tsan14+0x517ca2)
#2 _cgo_release_context <null> (tsan14+0x5176b0)
#3 go_callback <null> (tsan14+0x5173e3)
SUMMARY: ThreadSanitizer: signal handler spoils errno /home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/sys_linux_amd64.s:388 in runtime.cgoSigtramp
==================
--- FAIL: TestTSAN/tsan14 (3.33s)