[Bug]: mo-server crash when removing segment files
Is there an existing issue for the same bug?
- [X] I have checked the existing issues.
Environment
- Version or commit-id (e.g. v0.1.0 or 8b23a93):e1e7d5c7e54b516e58d23a5c90ea885576525ca6
- Hardware parameters:
- OS type:centos macos
- Others:
Actual Behavior
The path of the segment is normal when it is created. After SoftDelete, the path of the segment is abnormal at the time of GC, causing the remove to fail.
Expected Behavior
No response
Steps to Reproduce
Execute bvt repeatedly.
rarely recurs.
Additional information
{"level":"INFO","time":"2022/06/28 21:37:15.125114 +0800","caller":"db/gcop.go:39","msg":"[GCBLK] | [Appendable]BLOCK[<0:23419-20740-20790-0-0>] | Removed"}
{"level":"INFO","time":"2022/06/28 21:37:15.125121 +0800","caller":"segmentio/segment.go:275","msg":"Destroying Driver 20740"}
{"level":"INFO","time":"2022/06/28 21:37:15.125138 +0800","caller":"segmentio/driver.go:163","msg":"Unmount Driver: z\ufffd\ufffd\u0003\ufffd\ufffd\ufffd?e/20740.seg"}
{"level":"DEBUG","time":"2022/06/28 21:37:15.125165 +0800","caller":"segmentio/driver.go:307","msg":" z\ufffd\ufffd\u0003\ufffd\ufffd\ufffd?e/20740.seg-0xc00020df00 | SegmentFile | Destroying | Null-1-11 | Log Level1 ffffffffffffffc0-ffffffffffffffff"}
{"level":"INFO","time":"2022/06/28 21:37:15.125195 +0800","caller":"db/gcop.go:67","msg":"[GCSEG] | [Appendable]SEGMENT[<0:23419-20740-0-0-0>] | BLKS=[<0:23419-20740-20790-0-0>,] | Removed"}
{"level":"INFO","time":"2022/06/28 21:37:15.125209 +0800","caller":"db/gcop.go:95","msg":"[GCTABLE] | TABLE[Op=SoftDelete][ID=23419][265681=>265693]<Index[86173:0/1]>[name=t1] | Ended:
goroutine 228 [running]: github.com/matrixorigin/matrixone/pkg/vm/engine/tae/dataio/segmentio.(*Driver).Destroy(0xc00020dea0) /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/dataio/segmentio/driver.go:176 +0x1b4 github.com/matrixorigin/matrixone/pkg/vm/engine/tae/dataio/segmentio.(*segmentFile).Destroy(0xc013604420) /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/dataio/segmentio/segment.go:285 +0x18f github.com/matrixorigin/matrixone/pkg/vm/engine/tae/dataio/segmentio.(*segmentFile).close(0xc0040b7800?) /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/dataio/segmentio/segment.go:272 +0x19 github.com/matrixorigin/matrixone/pkg/vm/engine/tae/common.(*RefHelper).Unref(0xc0040b7800?) /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/common/refs.go:50 +0x36 github.com/matrixorigin/matrixone/pkg/vm/engine/tae/tables.(*dataSegment).Destory(0xc013955ef0?) /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/tables/segment.go:60 +0x4f github.com/matrixorigin/matrixone/pkg/vm/engine/tae/catalog.(*SegmentEntry).DestroyData(...) /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/catalog/segment.go:388 github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db.gcSegmentClosure.func1() /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/db/gcop.go:81 +0x2bc github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db.gcTableClosure.func1() /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/db/gcop.go:102 +0x23c github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db.gcDatabaseClosure.func1() /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/db/gcop.go:124 +0x1f0 github.com/matrixorigin/matrixone/pkg/vm/engine/tae/tasks.(*FnTask).Execute(0xc000280180?) /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/tasks/types.go:122 +0x1b github.com/matrixorigin/matrixone/pkg/vm/engine/tae/tasks/ops.(*Op).OnExec(0xc005670b60) /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/tasks/ops/ops.go:90 +0x6f github.com/matrixorigin/matrixone/pkg/vm/engine/tae/tasks/worker.(*OpWorker).onOp(0xc0003df6c0, {0x7fe0d48ed998, 0xc0174005a0}) /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/tasks/worker/worker.go:214 +0x30 github.com/matrixorigin/matrixone/pkg/vm/engine/tae/tasks/worker.(*OpWorker).Start.func1() /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/tasks/worker/worker.go:144 +0xad created by github.com/matrixorigin/matrixone/pkg/vm/engine/tae/tasks/worker.(*OpWorker).Start /mnt/vdb/sjw/tae/matrixone/pkg/vm/engine/tae/tasks/worker/worker.go:136 +0xb6