emitter icon indicating copy to clipboard operation
emitter copied to clipboard

Published 20 hours ago • verified publisher icon emitter-io

panic: runtime error: invalid memory address or nil pointer dereference

Open ainilili opened this issue 4 years ago • 8 comments

We found a problem in use. The reason for the error is as follows. It should be the Close() function of the connection triggered by the write-down.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xe88c72]

goroutine 279179 [running]:
github.com/weaveworks/mesh.(*gossipSender).Broadcast(0xc0003cdd60, 0x62a3a158a6a5, 0x15191a0, 0xc01a97e590)
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip.go:202 +0xb2
github.com/weaveworks/mesh.(*gossipChannel).relayBroadcast(0xc0000ca380, 0x62a3a158a6a5, 0x15191a0, 0xc01a97e590)
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip_channel.go:116 +0x10b
github.com/weaveworks/mesh.(*gossipChannel).GossipBroadcast(0xc0000ca380, 0x15191a0, 0xc01a97e590)
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip_channel.go:83 +0x4f
github.com/emitter-io/emitter/internal/broker/cluster.(*Swarm).NotifyEndOf(0xc0000fa000, 0x15263e0, 0xc021b7da40)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/cluster/swarm.go:356 +0xa7
github.com/emitter-io/emitter/internal/broker.(*Service).notifyUnsubscribe(0xc0002f6000, 0xc021b7da40)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/service.go:304 +0x5f
github.com/emitter-io/emitter/internal/broker.(*Service).Unsubscribe(0xc0002f6000, 0x15262a0, 0xc0075be300, 0xc021b7da40, 0x63)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/service.go:417 +0x143
github.com/emitter-io/emitter/internal/broker.(*Conn).Close(0xc0075be300, 0xc00e67d2c0, 0x10000)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/conn.go:311 +0x21f
github.com/emitter-io/emitter/internal/broker.(*Conn).Process(0xc0075be300, 0x1511220, 0xc015775d40)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/conn.go:131 +0x23a
created by github.com/emitter-io/emitter/internal/broker.(*Service).onAcceptConn
	/go-build/src/github.com/emitter-io/emitter/internal/broker/service.go:311 +0x6e

I'm running a docker Hub lastest version. I don't know if the latest version has this problem, but it could cause a service outage!

ainilili avatar Jun 18 '20 03:06 ainilili

Thanks for the report, we'll fix it in the next version.

kelindar avatar Jun 18 '20 04:06 kelindar

Have a similar issue:

2021/01/20 10:01:28 [closing] panic recovered: runtime error: invalid memory address or nil pointer dereference 
 goroutine 10981 [running]:
runtime/debug.Stack(0xc00f494a88, 0x11a12c0, 0x1b31020)
	/usr/local/go/src/runtime/debug/stack.go:24 +0x9d
github.com/emitter-io/emitter/internal/broker.(*Conn).Close(0xc00964d680, 0x0, 0x0)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/conn.go:353 +0x32d
panic(0x11a12c0, 0x1b31020)
	/usr/local/go/src/runtime/panic.go:969 +0x166
github.com/weaveworks/mesh.(*gossipSender).Broadcast(0xc0042cd6d0, 0xbe249f1f5be4, 0x1525020, 0xc00ab7e720)
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip.go:202 +0xb2
github.com/weaveworks/mesh.(*gossipChannel).relayBroadcast(0xc0000d24c0, 0xbe249f1f5be4, 0x1525020, 0xc00ab7e720)
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip_channel.go:116 +0x10b
github.com/weaveworks/mesh.(*gossipChannel).GossipBroadcast(0xc0000d24c0, 0x1525020, 0xc00ab7e720)
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip_channel.go:83 +0x4f
github.com/emitter-io/emitter/internal/service/cluster.(*Swarm).Notify(0xc0003d4000, 0x1532520, 0xc007289300, 0x1)
	/go-build/src/github.com/emitter-io/emitter/internal/service/cluster/swarm.go:359 +0xb5
github.com/emitter-io/emitter/internal/broker.(*Conn).onConnect(0xc00964d680, 0xc006e94a00, 0x4)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/conn.go:344 +0x1a6
github.com/emitter-io/emitter/internal/broker.(*Conn).onReceive(0xc00964d680, 0x15325e0, 0xc006e94a00, 0x0, 0x0)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/conn.go:194 +0x7aa
github.com/emitter-io/emitter/internal/broker.(*Conn).Process(0xc00964d680, 0x0, 0x0)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/conn.go:180 +0x1cb
created by github.com/emitter-io/emitter/internal/broker.(*Service).onAcceptConn
	/go-build/src/github.com/emitter-io/emitter/internal/broker/service.go:315 +0x6e

L3o-pold avatar Jan 20 '21 14:01 L3o-pold

so?, what to do finally?

beslnet avatar Jan 25 '21 18:01 beslnet

Scale at only one broker seems to work...

L3o-pold avatar Jan 25 '21 18:01 L3o-pold

Still an issue on 3.0

2022/02/01 19:16:29 [closing] panic recovered: runtime error: invalid memory address or nil pointer dereference 
 goroutine 4704 [running]:
runtime/debug.Stack()
	/usr/local/go/src/runtime/debug/stack.go:24 +0x65
github.com/emitter-io/emitter/internal/broker.(*Conn).Close(0xc00cb79680)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/conn.go:353 +0x66
panic({0x1162660, 0x19ca0f0})
	/usr/local/go/src/runtime/panic.go:1038 +0x215
github.com/weaveworks/mesh.(*gossipSender).Broadcast(0xc0000e2410, 0x1412698, {0x1402a90, 0xc00880f340})
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip.go:202 +0x11c
github.com/weaveworks/mesh.(*gossipChannel).relayBroadcast(0xc0001244c0, 0x1c, {0x1402a90, 0xc00880f340})
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip_channel.go:116 +0xc6
github.com/weaveworks/mesh.(*gossipChannel).GossipBroadcast(0xc00880f340, {0x1402a90, 0xc00880f340})
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip_channel.go:83 +0x37
github.com/emitter-io/emitter/internal/service/cluster.(*Swarm).Notify(0xc0005cee00, {0x1404e28, 0xc00b29b450}, 0x1)
	/go-build/src/github.com/emitter-io/emitter/internal/service/cluster/swarm.go:359 +0xbd
github.com/emitter-io/emitter/internal/broker.(*Service).NotifySubscribe(0xc0000f00d0, {0x1404d98, 0xc00cb79680}, 0xc00b29b450)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/service.go:292 +0xc5
github.com/emitter-io/emitter/internal/service/pubsub.(*Service).Subscribe(0xc00032a280, {0x1404d98, 0xc00cb79680}, 0xc00b29b450)
	/go-build/src/github.com/emitter-io/emitter/internal/service/pubsub/subscribe.go:38 +0xb5
github.com/emitter-io/emitter/internal/service/pubsub.(*Service).OnSubscribe(0xc00032a280, {0x1420190, 0xc00cb79680}, {0xc00c706de4, 0x8ccc85, 0x7781ed})
	/go-build/src/github.com/emitter-io/emitter/internal/service/pubsub/subscribe.go:70 +0x3ab
github.com/emitter-io/emitter/internal/broker.(*Conn).onReceive(0xc00cb79680, {0x1405068, 0xc007a4be60})
	/go-build/src/github.com/emitter-io/emitter/internal/broker/conn.go:213 +0x7e5
github.com/emitter-io/emitter/internal/broker.(*Conn).Process(0xc00cb79680)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/conn.go:179 +0x278
created by github.com/emitter-io/emitter/internal/broker.(*Service).onAcceptConn
	/go-build/src/github.com/emitter-io/emitter/internal/broker/service.go:321 +0x72

L3o-pold avatar Feb 01 '22 19:02 L3o-pold

With quite a bit of delay, @L3o-pold can you check whether your issue is still present on ´:latest´?

#417 Might correct it.

Florimond avatar Dec 18 '23 19:12 Florimond

@Florimond still an issue in 3.1

panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xf08592]

goroutine 272 [running]:
github.com/weaveworks/mesh.(*gossipSender).Broadcast(0xc0005b08c0, 0x19d7fe0?, {0x19d32a8?, 0xc009a1a4b0})
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip.go:202 +0x112
github.com/weaveworks/mesh.(*gossipChannel).relayBroadcast(0xc0005a68c0, 0x0?, {0x19d32a8, 0xc009a1a4b0})
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip_channel.go:116 +0x10a
github.com/weaveworks/mesh.(*gossipChannel).GossipBroadcast(0xc009a1a4b0?, {0x19d32a8?, 0xc009a1a4b0?})
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip_channel.go:83 +0x31
github.com/emitter-io/emitter/internal/service/cluster.(*Swarm).Notify(0xc0005a8070, {0x19d54d0, 0xc008594c80}, 0x0)
	/go-build/src/github.com/emitter-io/emitter/internal/service/cluster/swarm.go:359 +0xb8
github.com/emitter-io/emitter/internal/broker.(*Service).NotifyUnsubscribe(0xc0004b4ea0, {0x19d5380?, 0xc000474510?}, 0xc008594c80)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/service.go:309 +0xb7
github.com/emitter-io/emitter/internal/service/pubsub.(*Service).Unsubscribe(0xc00047c180, {0x19d5380, 0xc000474510}, 0xc008594c80)
	/go-build/src/github.com/emitter-io/emitter/internal/service/pubsub/unsubscribe.go:37 +0xf8
github.com/emitter-io/emitter/internal/broker.(*Conn).Close(0xc000474510)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/conn.go:359 +0x1c6
panic({0x1468820?, 0x216a1c0?})
	/usr/local/go/src/runtime/panic.go:914 +0x21f
github.com/weaveworks/mesh.(*gossipSender).Broadcast(0xc0005b08c0, 0x19d7fe0?, {0x19d32a8?, 0xc0004789c0})
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip.go:202 +0x112
github.com/weaveworks/mesh.(*gossipChannel).relayBroadcast(0xc0005a68c0, 0x1c?, {0x19d32a8, 0xc0004789c0})
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip_channel.go:116 +0x10a
github.com/weaveworks/mesh.(*gossipChannel).GossipBroadcast(0xc0004789c0?, {0x19d32a8?, 0xc0004789c0?})
	/go/pkg/mod/github.com/weaveworks/[email protected]/gossip_channel.go:83 +0x31
github.com/emitter-io/emitter/internal/service/cluster.(*Swarm).Notify(0xc0005a8070, {0x19d54d0, 0xc008594b90}, 0x1)
	/go-build/src/github.com/emitter-io/emitter/internal/service/cluster/swarm.go:359 +0xb8
github.com/emitter-io/emitter/internal/broker.(*Service).NotifySubscribe(0xc0004b4ea0, {0x19d5380?, 0xc000474510?}, 0xc008594b90)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/service.go:294 +0xba
github.com/emitter-io/emitter/internal/service/pubsub.(*Service).Subscribe(0xc00047c180, {0x19d5380, 0xc000474510}, 0xc008594b90)
	/go-build/src/github.com/emitter-io/emitter/internal/service/pubsub/subscribe.go:39 +0xb1
github.com/emitter-io/emitter/internal/service/pubsub.(*Service).OnSubscribe(0xc00047c180, {0x19dd418, 0xc000474510}, {0xc000504c04?, 0x54?, 0xc000514e58?})
	/go-build/src/github.com/emitter-io/emitter/internal/service/pubsub/subscribe.go:71 +0x334
github.com/emitter-io/emitter/internal/broker.(*Conn).onReceive(0xc000474510, {0x19d5410, 0xc006ef56c0})
	/go-build/src/github.com/emitter-io/emitter/internal/broker/conn.go:213 +0x6ed
github.com/emitter-io/emitter/internal/broker.(*Conn).Process(0xc000474510)
	/go-build/src/github.com/emitter-io/emitter/internal/broker/conn.go:179 +0x258
created by github.com/emitter-io/emitter/internal/broker.(*Service).onAcceptConn in goroutine 270
	/go-build/src/github.com/emitter-io/emitter/internal/broker/service.go:323 +0x66

L3o-pold avatar Mar 22 '24 17:03 L3o-pold

@L3o-pold Ok then, probably I didn't understand the issue to begin with.

What I thought, is that it was an issue when forming a cluster, issue that I fixed. And to test it I followed this procedure...

What I do is that I launch a server in vscode with the following config:

{
    "listen": ":8080",
    "license": "PfA8IOPQD29mo7STD_-9g2DhFhDyBq2rfRjU3oqa8yi6O8DAEHZtFblFg9Vc3-XW-nDpW2aivIn7CJbRmpUOAQ:3",
	"tls": {
		"listen": ":443"
	},
	"cluster": {
		"listen": ":4000",
		"advertise": "external:4000",
		"seed": ":4001"
	},
	"storage": {
		"provider": "inmemory"
	}
}

And I launch a docker of the same version (v3.1) of emitter with:

  • the same license key
  • port 4001:4000
  • port 8081:8080

I have the following message on the vscode Emitter when it starts:

2024/03/26 06:00:24 [swarm] joining (0.0.0.0:4001)

And the following message in the docker logs:

2024-03-26 06:00:24 2024/03/26 05:00:24 [swarm] peer created (00:93:37:5f:67:26)

Then I use the little sample that comes with the python SDK to test the publish on a channel when connect to either instance.

Now, if the issue reported here is different, do you mind clarifying for me, as I'm a bit lost? Thanks a lot.

Florimond avatar Mar 26 '24 05:03 Florimond