wails icon indicating copy to clipboard operation
wails copied to clipboard

Published 20 hours ago • verified publisher icon wailsapp

Fatal error upon executing a gRPC request in Linux

Open sdzyba opened this issue 3 years ago • 9 comments

Description

Wails application crashes when executing a gRPC request. The problem seems to happen only on Linux, on macOS the requests are executed well. Also plain non-gRPC HTTP requests are executed well on any kind of a system.

signal 11 received but handler not on signal stack
fatal error: non-Go code set up signal handler without SA_ONSTACK flag

runtime stack:
runtime: unexpected return pc for runtime.sigtramp called from 0x7f33e7224327
stack: frame={sp:0xc0003a4050, fp:0xc0003a40a0} stack=[0xc00039bf48,0xc0003a4348)
0x000000c0003a3f50:  0x000000c0003a3f88  0x000000c0003a3fb0
0x000000c0003a3f60:  0x0000000000000000  0x00007f33e0a56108
0x000000c0003a3f70:  0x0000000000000280  0x000000c0003e6800
0x000000c0003a3f80:  0x000000c0000dcc80  0x000000c000410000
0x000000c0003a3f90:  0x0000000000000000  0x0000000000008000
0x000000c0003a3fa0:  0x000000c0003a4018  0x000000000044f3c5 <runtime.sigtrampgo+0x0000000000000145>
0x000000c0003a3fb0:  0x000000c00000000b  0x000000c0003e6800
0x000000c0003a3fc0:  0x000000c0003a3fd8  0x0000000000000280
0x000000c0003a3fd0:  0x0000000000000006  0x0000000000000000
0x000000c0003a3fe0:  0x0000000000000000  0x0000000000000000
0x000000c0003a3ff0:  0x0000000000000000  0x0000000000000000
0x000000c0003a4000:  0x000000c0003e5860  0x000000c0003a42f0
0x000000c0003a4010:  0x000000c0003a41c0  0x000000c0003a4040
0x000000c0003a4020:  0x000000000046d2ce <runtime.sigtrampgo+0x000000000000002e>  0x000000000000000b
0x000000c0003a4030:  0x000000c0003a42f0  0x000000c0003a41c0
0x000000c0003a4040:  0x000000c0003a4090  0x000000000046c77d <runtime.sigtramp+0x000000000000003d>
0x000000c0003a4050: <0x000000000000000b  0x000000c0003a42f0
0x000000c0003a4060:  0x000000c0003a41c0  0x000000c0003a41e8
0x000000c0003a4070:  0x000000c0003a40d8  0x0000000000000000
0x000000c0003a4080:  0x0000000000000000  0x0000000000000005
0x000000c0003a4090:  0x00007f33e760a0a0 !0x00007f33e7224327
0x000000c0003a40a0: >0x0000000000b81860  0x0000000001496c20
0x000000c0003a40b0:  0x000000c0003a41c0  0x00007f33e760a000
0x000000c0003a40c0:  0x000000c0003a42f0  0x0000000b0000000b
0x000000c0003a40d0:  0x00000000005063f5 <bytes.(*Buffer).WriteString+0x00000000000000b5>  0x0000000000000000
0x000000c0003a40e0:  0x000000090cbddb26  0x0000000000000002
0x000000c0003a40f0:  0x0000000000000007  0x000000000000000d
0x000000c0003a4100:  0x000000000000000f  0x000000c0003a4160
0x000000c0003a4110:  0x0000000000000100  0x0000000000000013
0x000000c0003a4120:  0xffffffffffffffff  0x000000090cbddb26
0x000000c0003a4130:  0xc0ac5f3b9df5168c  0x0000000000413fc5 <runtime.mallocgc+0x00000000000006c5>
0x000000c0003a4140:  0x0000000000000016  0x000000c00036b830
0x000000c0003a4150:  0x00007f33e0a60b20  0x0000000000000005
0x000000c0003a4160:  0x0001000000000000  0x00007f33b859e2b0
0x000000c0003a4170:  0x0000000000000000  0xe9bf890924c38400
0x000000c0003a4180:  0x0000000000000200  0x0000000000000000
0x000000c0003a4190:  0x000000c0003a48d8  0x000000c0003a47f0
runtime.throw({0xcdb897?, 0xc000034400?})
  /snap/go/9848/src/runtime/panic.go:992 +0x71
runtime.sigNotOnStack(0xb)
  /snap/go/9848/src/runtime/signal_unix.go:1002 +0x65
runtime.adjustSignalStack(0xb, 0xc0003e6800, 0xc0003a3fd8)
  /snap/go/9848/src/runtime/signal_unix.go:577 +0x290
runtime.sigtrampgo(0xb, 0xc0003a42f0, 0xc0003a41c0)
  /snap/go/9848/src/runtime/signal_unix.go:467 +0x145
runtime.sigtrampgo(0xb, 0xc0003a42f0, 0xc0003a41c0)
  <autogenerated>:1 +0x2e
runtime: unexpected return pc for runtime.sigtramp called from 0x7f33e7224327
stack: frame={sp:0xc0003a4050, fp:0xc0003a40a0} stack=[0xc00039bf48,0xc0003a4348)
0x000000c0003a3f50:  0x000000c0003a3f88  0x000000c0003a3fb0
0x000000c0003a3f60:  0x0000000000000000  0x00007f33e0a56108
0x000000c0003a3f70:  0x0000000000000280  0x000000c0003e6800
0x000000c0003a3f80:  0x000000c0000dcc80  0x000000c000410000
0x000000c0003a3f90:  0x0000000000000000  0x0000000000008000
0x000000c0003a3fa0:  0x000000c0003a4018  0x000000000044f3c5 <runtime.sigtrampgo+0x0000000000000145>
0x000000c0003a3fb0:  0x000000c00000000b  0x000000c0003e6800
0x000000c0003a3fc0:  0x000000c0003a3fd8  0x0000000000000280
0x000000c0003a3fd0:  0x0000000000000006  0x0000000000000000
0x000000c0003a3fe0:  0x0000000000000000  0x0000000000000000
0x000000c0003a3ff0:  0x0000000000000000  0x0000000000000000
0x000000c0003a4000:  0x000000c0003e5860  0x000000c0003a42f0
0x000000c0003a4010:  0x000000c0003a41c0  0x000000c0003a4040
0x000000c0003a4020:  0x000000000046d2ce <runtime.sigtrampgo+0x000000000000002e>  0x000000000000000b
0x000000c0003a4030:  0x000000c0003a42f0  0x000000c0003a41c0
0x000000c0003a4040:  0x000000c0003a4090  0x000000000046c77d <runtime.sigtramp+0x000000000000003d>
0x000000c0003a4050: <0x000000000000000b  0x000000c0003a42f0
0x000000c0003a4060:  0x000000c0003a41c0  0x000000c0003a41e8
0x000000c0003a4070:  0x000000c0003a40d8  0x0000000000000000
0x000000c0003a4080:  0x0000000000000000  0x0000000000000005
0x000000c0003a4090:  0x00007f33e760a0a0 !0x00007f33e7224327
0x000000c0003a40a0: >0x0000000000b81860  0x0000000001496c20
0x000000c0003a40b0:  0x000000c0003a41c0  0x00007f33e760a000
0x000000c0003a40c0:  0x000000c0003a42f0  0x0000000b0000000b
0x000000c0003a40d0:  0x00000000005063f5 <bytes.(*Buffer).WriteString+0x00000000000000b5>  0x0000000000000000
0x000000c0003a40e0:  0x000000090cbddb26  0x0000000000000002
0x000000c0003a40f0:  0x0000000000000007  0x000000000000000d
0x000000c0003a4100:  0x000000000000000f  0x000000c0003a4160
0x000000c0003a4110:  0x0000000000000100  0x0000000000000013
0x000000c0003a4120:  0xffffffffffffffff  0x000000090cbddb26
0x000000c0003a4130:  0xc0ac5f3b9df5168c  0x0000000000413fc5 <runtime.mallocgc+0x00000000000006c5>
0x000000c0003a4140:  0x0000000000000016  0x000000c00036b830
0x000000c0003a4150:  0x00007f33e0a60b20  0x0000000000000005
0x000000c0003a4160:  0x0001000000000000  0x00007f33b859e2b0
0x000000c0003a4170:  0x0000000000000000  0xe9bf890924c38400
0x000000c0003a4180:  0x0000000000000200  0x0000000000000000
0x000000c0003a4190:  0x000000c0003a48d8  0x000000c0003a47f0
runtime.sigtramp()
  /snap/go/9848/src/runtime/sys_linux_amd64.s:361 +0x3d

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
  /snap/go/9848/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc00006afe8 sp=0xc00006afe0 pc=0x46ab81

goroutine 1 [syscall, locked to thread]:
github.com/wailsapp/wails/v2/internal/frontend/desktop/linux._Cfunc_gtk_main()
  _cgo_gotypes.go:966 +0x45
github.com/wailsapp/wails/v2/internal/frontend/desktop/linux.(*Window).Run(0xc000333f80, {0xc00034b890, 0x16})
  /home/void/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/frontend/desktop/linux/window.go:807 +0xd3
github.com/wailsapp/wails/v2/internal/frontend/desktop/linux.(*Frontend).Run(0xc0003659e0, {0xf1e0f0?, 0xc000379020?})
  /home/void/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/frontend/desktop/linux/frontend.go:138 +0xd5
github.com/wailsapp/wails/v2/internal/frontend/devserver.(*DevWebServer).Run(0xc000330f30, {0xf1e0f0?, 0xc000379020})
  /home/void/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/frontend/devserver/devserver.go:135 +0x787
github.com/wailsapp/wails/v2/internal/appng.(*App).Run(0xc0002ef900)
  /home/void/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/appng/app_dev.go:55 +0x31
github.com/wailsapp/wails/v2.Run(0xc00031d200?)
  /home/void/go/pkg/mod/github.com/wailsapp/wails/[email protected]/wails.go:40 +0xe5
main.main()
  /home/void/Documents/multibase/main.go:28 +0x48a

goroutine 6 [chan receive]:
github.com/wailsapp/wails/v2/internal/frontend/desktop/linux.(*Frontend).startMessageProcessor(0x0?)
  /home/void/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/frontend/desktop/linux/frontend.go:107 +0x59
created by github.com/wailsapp/wails/v2/internal/frontend/desktop/linux.NewFrontend
  /home/void/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/frontend/desktop/linux/frontend.go:93 +0x2f6

goroutine 8 [syscall]:
os/signal.signal_recv()
  /snap/go/9848/src/runtime/sigqueue.go:151 +0x2f
os/signal.loop()
  /snap/go/9848/src/os/signal/signal_unix.go:23 +0x19
created by os/signal.Notify.func1.1
  /snap/go/9848/src/os/signal/signal.go:151 +0x2a

goroutine 9 [chan receive]:
github.com/wailsapp/wails/v2/internal/signal.Start.func1()
  /home/void/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/signal/signal.go:31 +0x26
created by github.com/wailsapp/wails/v2/internal/signal.Start
  /home/void/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/signal/signal.go:29 +0x91

goroutine 16 [IO wait]:
internal/poll.runtime_pollWait(0x7f33b85d8748, 0x72)
  /snap/go/9848/src/runtime/netpoll.go:302 +0x89
internal/poll.(*pollDesc).wait(0xc0003f2900?, 0xc0003e4680?, 0x0)
  /snap/go/9848/src/internal/poll/fd_poll_runtime.go:83 +0x32
internal/poll.(*pollDesc).waitRead(...)
  /snap/go/9848/src/internal/poll/fd_poll_runtime.go:88
internal/poll.(*FD).Accept(0xc0003f2900)
  /snap/go/9848/src/internal/poll/fd_unix.go:614 +0x22c
net.(*netFD).accept(0xc0003f2900)
  /snap/go/9848/src/net/fd_unix.go:172 +0x35
net.(*TCPListener).accept(0xc00034e9c0)
  /snap/go/9848/src/net/tcpsock_posix.go:139 +0x28
net.(*TCPListener).AcceptTCP(0xc00034e9c0)
  /snap/go/9848/src/net/tcpsock.go:275 +0x3d
github.com/labstack/echo/v4.tcpKeepAliveListener.Accept({0x445a20?})
  /home/void/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:941 +0x1d
net/http.(*Server).Serve(0xc000164fc0, {0xf1ced0, 0xc0003726f0})
  /snap/go/9848/src/net/http/server.go:3039 +0x385
github.com/labstack/echo/v4.(*Echo).Start(0xc00034cb40, {0xc00003e1aa, 0xf})
  /home/void/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:647 +0xce
github.com/wailsapp/wails/v2/internal/frontend/devserver.(*DevWebServer).Run.func4(0xc000045560?, 0x0?)
  /home/void/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/frontend/devserver/devserver.go:117 +0x36
created by github.com/wailsapp/wails/v2/internal/frontend/devserver.(*DevWebServer).Run
  /home/void/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/frontend/devserver/devserver.go:116 +0x69b

goroutine 138 [runnable]:
runtime.Gosched(...)
  /snap/go/9848/src/runtime/proc.go:317
google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0003648a0)
  /home/void/go/pkg/mod/google.golang.org/[email protected]/internal/transport/controlbuf.go:571 +0x1f6
google.golang.org/grpc/internal/transport.newHTTP2Client.func3()
  /home/void/go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_client.go:415 +0x65
created by google.golang.org/grpc/internal/transport.newHTTP2Client
  /home/void/go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_client.go:413 +0x1f91

goroutine 137 [IO wait]:
internal/poll.runtime_pollWait(0x7f33b85d7de8, 0x72)
  /snap/go/9848/src/runtime/netpoll.go:302 +0x89
internal/poll.(*pollDesc).wait(0xc0001a7c80?, 0xc0003a6000?, 0x0)
  /snap/go/9848/src/internal/poll/fd_poll_runtime.go:83 +0x32
internal/poll.(*pollDesc).waitRead(...)
  /snap/go/9848/src/internal/poll/fd_poll_runtime.go:88
internal/poll.(*FD).Read(0xc0001a7c80, {0xc0003a6000, 0x8000, 0x8000})
  /snap/go/9848/src/internal/poll/fd_unix.go:167 +0x25a
net.(*netFD).Read(0xc0001a7c80, {0xc0003a6000?, 0x413fc5?, 0x3?})
  /snap/go/9848/src/net/fd_posix.go:55 +0x29
net.(*conn).Read(0xc000372a40, {0xc0003a6000?, 0x30?, 0x10401?})
  /snap/go/9848/src/net/net.go:183 +0x45
bufio.(*Reader).Read(0xc0003647e0, {0xc0000e0ac0, 0x9, 0xc0003e6800?})
  /snap/go/9848/src/bufio/bufio.go:236 +0x1b4
io.ReadAtLeast({0xf17460, 0xc0003647e0}, {0xc0000e0ac0, 0x9, 0x9}, 0x9)
  /snap/go/9848/src/io/io.go:331 +0x9a
io.ReadFull(...)
  /snap/go/9848/src/io/io.go:350
golang.org/x/net/http2.readFrameHeader({0xc0000e0ac0?, 0x9?, 0xc000378e40?}, {0xf17460?, 0xc0003647e0?})
  /home/void/go/pkg/mod/golang.org/x/[email protected]/http2/frame.go:237 +0x6e
golang.org/x/net/http2.(*Framer).ReadFrame(0xc0000e0a80)
  /home/void/go/pkg/mod/golang.org/x/[email protected]/http2/frame.go:498 +0x95
google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc000001a40)
  /home/void/go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_client.go:1498 +0x414
created by google.golang.org/grpc/internal/transport.newHTTP2Client
  /home/void/go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_client.go:365 +0x193f

goroutine 133 [running]:
  goroutine running on other thread; stack unavailable
created by github.com/wailsapp/wails/v2/internal/frontend/desktop/linux.(*Frontend).processMessage
  /home/void/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/frontend/desktop/linux/frontend.go:259 +0x10c

goroutine 134 [select]:
google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000238000)
  /home/void/go/pkg/mod/google.golang.org/[email protected]/balancer_conn_wrappers.go:112 +0x73
created by google.golang.org/grpc.newCCBalancerWrapper
  /home/void/go/pkg/mod/google.golang.org/[email protected]/balancer_conn_wrappers.go:73 +0x22a

To Reproduce

  1. Create a Wails v2 application with a gRPC client.
  2. Build and launch Wails application in Ubuntu.
  3. Execute gRPC request.
  4. Application crashes.

Expected behaviour

gRPC request is executed without a crash.

System Details

System
------
OS:		Ubuntu
Version: 	22.04
ID:		ubuntu
Go Version:	go1.18.3
Platform:	linux
Architecture:	amd64

Wails
------
Version: 		v2.0.0-beta.38
Package Manager: 	apt

Dependency	Package Name		Status		Version
----------	------------		------		-------
*docker		docker.io		Available	20.10.12-0ubuntu4
gcc		build-essential		Installed	12.9ubuntu3
libgtk-3	libgtk-3-dev		Installed	3.24.33-1ubuntu2
libwebkit	libwebkit2gtk-4.0-dev	Installed	2.36.3-0ubuntu0.22.04.1
npm		npm			Installed	8.11.0
*nsis		nsis			Available	3.08-2
pkg-config	pkg-config		Installed	0.29.2-1ubuntu3

sdzyba avatar Jul 15 '22 07:07 sdzyba

Thanks for opening this. This looks like your code is emitting a SIGSEGV and Go is indicating that a different signal handler is installed. I'm not convinced it's a Wails issue at this point. Reference: https://stackoverflow.com/questions/56492105/how-to-set-sa-onstack-flag

leaanthony avatar Jul 15 '22 07:07 leaanthony

Thanks for the reference! The thing is that the very same code executed outside the Wails application works completely well. For instance this example.

I'm not sure but it looks the issue is either on Wails' or webkit's (webkitgtk?) side. I've tried to search if it's a webkit but no luck, unfortunately.

sdzyba avatar Jul 15 '22 07:07 sdzyba

Just be aware that most code in Wails does not run on the main thread (most code is executed in Goroutines) so that might be something to look at.

leaanthony avatar Jul 15 '22 07:07 leaanthony

So a little more details: The issue is experienced in my Ubuntu virtual machine, where I port-forwarding the connection from a remote Kubernetes cluster and trying to make a request to.

So I tried to narrow the issue by trying various separated scenarios. If I execute the gRPC request to a Kubernetes cluster with the same code but outside the Wails app, it works. If I run the gRPC server on a localhost instead of port-forwarding the kubernetes' server, it works as well, my Wails app doesn't crash and request is executed successfully.

So I'm not completely sure but it seems that the problem is lying somewhere on an intersection between Wails runtime and the Kubernetes port-forwarding mechanism. But it happens only to gRPC connections, regular HTTP works well even with port-forwarding.

The interesting thing is that there is another gRPC client built on Wails v1: https://github.com/rogchap/wombat I tried it and it works absolutely well, although from what I see in the codebase it uses the same grpc library for connections and there are no any tricky options/workarounds used.

I, on contrary, use Wails v2. Maybe I should try Wails v1 as well, not sure.

sdzyba avatar Jul 17 '22 16:07 sdzyba

Which part of the runtime do you think is interfering with the grpc calls?

leaanthony avatar Jul 17 '22 21:07 leaanthony

No clue, unfortunately. I've also tried to execute the grpc call not during Wails request processing but for instance on App startup stage: the same result.

Also found a somewhat similar issue in goqt, well at least the error message is quite the same except for the signal number, signal 11 in my message is a segmentation violation. Maybe gtk or webkit causes a similar problem as described here.

Other than that to be honest I'm run out of ideas of what to do next.

sdzyba avatar Jul 17 '22 22:07 sdzyba

If port forwarding works but direct access doesn't, sounds like a security issue to me. Shot in the dark: on Mac there's this workaround for a similar issue: https://wails.io/docs/next/guides/troubleshooting#mac. Maybe something worth investigating as the only apparent difference is the use of a host name. Is your DNS ok?

leaanthony avatar Jul 18 '22 09:07 leaanthony

It's the opposite: direct access works, crash with port-forwarding. Also the issue is reproduced not only on my machine, friend of mine from another country got the same results in his Ubuntu virtualbox image.

sdzyba avatar Jul 24 '22 17:07 sdzyba

At this point I would suggest compiling with wails build -debug then running through gdb to get a better understanding of where the error is happening

leaanthony avatar Jul 24 '22 21:07 leaanthony

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Nov 02 '22 02:11 stale[bot]

This should now be fixed on master 👍

leaanthony avatar Nov 30 '22 20:11 leaanthony

@leaanthony ah that's great to hear. Thanks a lot, will try it out. :)

sdzyba avatar Nov 30 '22 21:11 sdzyba

Sorry it took so long! 😫

leaanthony avatar Nov 30 '22 21:11 leaanthony