Panic when using grpc proxying

[sbulman]

Describe the bug

An app that is using grpc proxying is panicking with the message below. It doesn't happen immediately, but consistently panics after a period of uptime. I am running the app in standalone mode at the moment and using the default redis components for storage and pubsub.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x2148353]

goroutine 1 [running]:
github.com/dapr/dapr/pkg/messaging/v1.(*InvokeMethodResponse).Status(...)
        /Users/runner/work/dapr/dapr/pkg/messaging/v1/invoke_method_response.go:99
github.com/dapr/dapr/pkg/runtime/pubsub.GetSubscriptionsHTTP({0x4c9aae0, 0xc0003ea4d0}, {0x4d56630, 0xc0005140f0})
        /Users/runner/work/dapr/dapr/pkg/runtime/pubsub/subscriptions.go:85 +0x2d3

To Reproduce

I have enabled grpc proxying in the default config and followed the hello world grpc-go example.

I have reflection enabled and use the grpc health server e.g.,

s := grpc.NewServer()
reflection.Register(s)
grpc_health_v1.RegisterHealthServer(s, health.NewServer())

Expected behavior

Understand this is a preview feature, but any help / fix would be much appreciated.

[sbulman]

A more complete trace of the panic

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x2148353]

goroutine 1 [running]:
github.com/dapr/dapr/pkg/messaging/v1.(*InvokeMethodResponse).Status(...)
        /Users/runner/work/dapr/dapr/pkg/messaging/v1/invoke_method_response.go:99
github.com/dapr/dapr/pkg/runtime/pubsub.GetSubscriptionsHTTP({0x4c9aae0, 0xc0004e8ee0}, {0x4d56630, 0xc00000c390})
        /Users/runner/work/dapr/dapr/pkg/runtime/pubsub/subscriptions.go:85 +0x2d3
github.com/dapr/dapr/pkg/runtime.(*DaprRuntime).getTopicRoutes(0xc00045f600)
        /Users/runner/work/dapr/dapr/pkg/runtime/runtime.go:1252 +0x170
github.com/dapr/dapr/pkg/runtime.(*DaprRuntime).beginPubSub(0xc00045f600, {0xc000ace4fa, 0xb}, {0x4cf3ed8, 0xc0005e8e00})
        /Users/runner/work/dapr/dapr/pkg/runtime/runtime.go:521 +0x150
github.com/dapr/dapr/pkg/runtime.(*DaprRuntime).startSubscribing(0xc00045f600)
        /Users/runner/work/dapr/dapr/pkg/runtime/runtime.go:2215 +0xa9
github.com/dapr/dapr/pkg/runtime.(*DaprRuntime).initRuntime(0xc00045f600, 0xc000791ee0)
        /Users/runner/work/dapr/dapr/pkg/runtime/runtime.go:456 +0x1049
github.com/dapr/dapr/pkg/runtime.(*DaprRuntime).Run(0xc00045f600, {0xc000d1f570, 0x8, 0x10})
        /Users/runner/work/dapr/dapr/pkg/runtime/runtime.go:265 +0x208
main.main()
        /Users/runner/work/dapr/dapr/cmd/daprd/main.go:170 +0x15ad

[cbovar]

Hello, I'm having the same problem. Were you able to solve it?

[yaron2]

@sbulman does this happen when receiving pubsub messages?

[sbulman]

@yaron2 from what I recall this wasn't related to sending / receiving messages. The app that was using gRPC proxying appeared to start up fine but then following a period of time started to panic with the above message.

[sbulman]

I just tried with the last version. The panic seems to have gone, I now just get an error logged

ERRO[0002] error occurred while beginning pubsub pubsub: error when reading response headers: read tcp4 127.0.0.1:60662->127.0.0.1:60000: read: connection reset by peer. Buffer size=15, contents: "\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x05\x00\x00@\x00"  app_id=XXX instance=C02Z3A3GLVCF scope=dapr.runtime type=log ver=1.7.0

I'm not sure if this helps.

[stannok]

I have the same issue. Is there any progress on this issue?

[stannok]

This was a mistake on my end!

I used the wrong annotation:

dapr.io/protocol: 'grpc'. This caused the app to default to the http protocol, which caused the headers error.

After switching to:

dapr.io/app-protocol: 'grpc' all is working as expected.