crawlab icon indicating copy to clipboard operation
crawlab copied to clipboard

MongoDB network issue results in Crawlab master node shutdown

Open tikazyq opened this issue 3 years ago • 0 comments

Describe the bug MongoDB network issue results in Crawlab master node shutdown.

Master container logs are as below.

crawlab_example_master | server selection error: server selection timeout, current topology: { Type: Single, Servers: [{ Addr: mongo:27017, Type: Unknown, Last error: connection(mongo:27017[-7]) unable to write wire message to network: write tcp 192.168.80.3:47772->192.168.80.2:27017: i/o timeout }, ] }
crawlab_example_master | /go/pkg/mod/github.com/crawlab-team/[email protected]/trace.go:13 github.com/crawlab-team/go-trace.TraceError()
crawlab_example_master | /go/pkg/mod/github.com/crawlab-team/[email protected]/models/service/binder_list.go:82 github.com/crawlab-team/crawlab-core/models/service.(*ListBinder).Process()
crawlab_example_master | /go/pkg/mod/github.com/crawlab-team/[email protected]/models/service/binder_list.go:44 github.com/crawlab-team/crawlab-core/models/service.(*ListBinder).Bind()
crawlab_example_master | /go/pkg/mod/github.com/crawlab-team/[email protected]/models/service/base_service.go:67 github.com/crawlab-team/crawlab-core/models/service.(*BaseService).GetList()
crawlab_example_master | /go/pkg/mod/github.com/crawlab-team/[email protected]/models/service/schedule_service.go:34 github.com/crawlab-team/crawlab-core/models/service.(*Service).GetScheduleList()
crawlab_example_master | /go/pkg/mod/github.com/crawlab-team/[email protected]/schedule/service.go:175 github.com/crawlab-team/crawlab-core/schedule.(*Service).fetch()
crawlab_example_master | /go/pkg/mod/github.com/crawlab-team/[email protected]/schedule/service.go:134 github.com/crawlab-team/crawlab-core/schedule.(*Service).update()
crawlab_example_master | /go/pkg/mod/github.com/crawlab-team/[email protected]/schedule/service.go:122 github.com/crawlab-team/crawlab-core/schedule.(*Service).Update()
crawlab_example_master | /usr/local/go/src/runtime/asm_amd64.s:1371 runtime.goexit()
crawlab_example_master | panic: runtime error: invalid memory address or nil pointer dereference
crawlab_example_master | [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xf1bf72]
crawlab_example_master |
crawlab_example_master | goroutine 140 [running]:
crawlab_example_master | github.com/crawlab-team/crawlab-core/models/service.(*Service).GetScheduleList(0xc00084cbe0, 0xc0747bf620, 0x0, 0x7, 0xc074840f28, 0x3fd0000000000000, 0x0, 0x2d353862342d3131)
crawlab_example_master | 	/go/pkg/mod/github.com/crawlab-team/[email protected]/models/service/schedule_service.go:35 +0x72
crawlab_example_master | github.com/crawlab-team/crawlab-core/schedule.(*Service).fetch(0xc00010e000, 0x49794d6a5a444d34, 0x49695a69356d4973)
crawlab_example_master | 	/go/pkg/mod/github.com/crawlab-team/[email protected]/schedule/service.go:175 +0xad
crawlab_example_master | github.com/crawlab-team/crawlab-core/schedule.(*Service).update(0xc00010e000)
crawlab_example_master | 	/go/pkg/mod/github.com/crawlab-team/[email protected]/schedule/service.go:134 +0x45
crawlab_example_master | github.com/crawlab-team/crawlab-core/schedule.(*Service).Update(0xc00010e000)
crawlab_example_master | 	/go/pkg/mod/github.com/crawlab-team/[email protected]/schedule/service.go:122 +0x28
crawlab_example_master | created by github.com/crawlab-team/crawlab-core/schedule.(*Service).Start
crawlab_example_master | 	/go/pkg/mod/github.com/crawlab-team/[email protected]/schedule/service.go:78 +0x51

To Reproduce Steps to reproduce the behavior:

  1. cut MongoDB network to Crawlab master node

Expected behavior Master should not shutdown even if MongoDB is not accessible.

Screenshots image

image

tikazyq avatar Jun 17 '22 05:06 tikazyq