cockroach ccl/changefeedccl: TestTableNameCollision failed

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2.10-rc @ 4c8ca4cf727684f1f2acd54da6ff4cf60fdf6587:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 1945355 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe2db20?)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc016698c00)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d6a738, 0xc011459400})
	github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d6a738, 0xc011459400}, 0x2, 0x3, 0x0, 0x3?, {0x66aa1b4, 0x21}, {0xc0029e3c70, 0x1, ...})
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
	github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc009c72460, {0x7d6a738?, 0xc011459400})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d6a738?, 0xc011459400?})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
	github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1945312
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96

Log preceding fatal error

=== RUN   TestTableNameCollision
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision1670933510
    test_log_scope.go:81: use -show-logs to present logs inline
=== RUN   TestTableNameCollision/kafka
    helpers_test.go:862: making server as system tenant
    helpers_test.go:937: making kafka feed factory
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection

Parameters:

TAGS=bazel,gss
stress=true

Help

Same failure on other branches

#128868 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc branch-release-23.2 release-blocker]

/cc @cockroachdb/cdc _{This test on roachdash | Improve this report!}

Jira issue: CRDB-41450

Aug 19 '24 06:08 cockroach-teamcity

I'm suspecting it might be related to the new Kafka sink given the stack trace and also the metamorphic constant:

initialized metamorphic constant "changefeed.new_kafka_sink.enabled" with value true

Assigning Miles to investigate.

Aug 19 '24 22:08 andyyang890

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2.10-rc @ 4c8ca4cf727684f1f2acd54da6ff4cf60fdf6587:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 1980818 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe2db20?)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc00a5fe400)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d6a738, 0xc01049f400})
	github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d6a738, 0xc01049f400}, 0x2, 0x3, 0x0, 0x3?, {0x66aa1b4, 0x21}, {0xc004af1c70, 0x1, ...})
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
	github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc008d2bea0, {0x7d6a738?, 0xc01049f400})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d6a738?, 0xc01049f400?})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
	github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1980758
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96

Log preceding fatal error

=== RUN   TestTableNameCollision
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision1887480395
    test_log_scope.go:81: use -show-logs to present logs inline
=== RUN   TestTableNameCollision/kafka
    helpers_test.go:862: making server as system tenant
    helpers_test.go:937: making kafka feed factory

Parameters:

TAGS=bazel,gss
stress=true

Help

Same failure on other branches

#128868 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc branch-release-23.2 release-blocker]

_{This test on roachdash | Improve this report!}

Aug 21 '24 06:08 cockroach-teamcity

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2.10-rc @ 4c8ca4cf727684f1f2acd54da6ff4cf60fdf6587:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2038867 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe2db20?)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc00444f800)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d6a738, 0xc00d68f1d0})
	github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d6a738, 0xc00d68f1d0}, 0x2, 0x3, 0x0, 0x3?, {0x66aa1b4, 0x21}, {0xc002b23c70, 0x1, ...})
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
	github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc0232bb180, {0x7d6a738?, 0xc00d68f1d0})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d6a738?, 0xc00d68f1d0?})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
	github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2038796
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96

Log preceding fatal error

=== RUN   TestTableNameCollision
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision3015736221
    test_log_scope.go:81: use -show-logs to present logs inline
=== RUN   TestTableNameCollision/kafka
    helpers_test.go:862: making server as system tenant
    helpers_test.go:937: making kafka feed factory
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection

Parameters:

TAGS=bazel,gss
stress=true

Help

Same failure on other branches

#128868 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc branch-release-23.2 release-blocker]

_{This test on roachdash | Improve this report!}

Aug 23 '24 06:08 cockroach-teamcity

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2.10-rc @ 7b9d20d87208db1ac6c24b68b73c742eb9b1168d:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 1928942 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe2db20?)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc01f745800)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d6a738, 0xc01da0e190})
	github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d6a738, 0xc01da0e190}, 0x2, 0x3, 0x0, 0x3?, {0x66aa1b4, 0x21}, {0xc002d6bc70, 0x1, ...})
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
	github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc01e967880, {0x7d6a738?, 0xc01da0e190})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d6a738?, 0xc01da0e190?})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
	github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1928921
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96

Log preceding fatal error

=== RUN   TestTableNameCollision
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision3765710908
    test_log_scope.go:81: use -show-logs to present logs inline
=== RUN   TestTableNameCollision/kafka
    helpers_test.go:862: making server as system tenant
    helpers_test.go:937: making kafka feed factory

Parameters:

TAGS=bazel,gss
stress=true

Help

Same failure on other branches

#128868 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc branch-release-23.2 release-blocker]

_{This test on roachdash | Improve this report!}

Aug 24 '24 06:08 cockroach-teamcity

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2.10-rc @ 7b9d20d87208db1ac6c24b68b73c742eb9b1168d:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 1966613 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe2db20?)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc00c88e400)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d6a738, 0xc0116218b0})
	github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d6a738, 0xc0116218b0}, 0x2, 0x3, 0x0, 0x3?, {0x66aa1b4, 0x21}, {0xc002b3bc70, 0x1, ...})
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
	github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc00be301c0, {0x7d6a738?, 0xc0116218b0})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d6a738?, 0xc0116218b0?})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
	github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1966581
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96

Log preceding fatal error

=== RUN   TestTableNameCollision
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision1373367123
    test_log_scope.go:81: use -show-logs to present logs inline
=== RUN   TestTableNameCollision/kafka
    helpers_test.go:862: making server as system tenant
    helpers_test.go:937: making kafka feed factory
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection

Parameters:

TAGS=bazel,gss
stress=true

Help

Same failure on other branches

#128868 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc branch-release-23.2 release-blocker]

_{This test on roachdash | Improve this report!}

Aug 27 '24 06:08 cockroach-teamcity

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2.10-rc @ 456de6170f2bc7af940df456a4470a665624c4b3:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2005138 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe2db20?)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc008282400)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d6a738, 0xc0091964b0})
	github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d6a738, 0xc0091964b0}, 0x2, 0x3, 0x0, 0x3?, {0x66aa1b4, 0x21}, {0xc0068b9c70, 0x1, ...})
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
	github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc00cf5bc00, {0x7d6a738?, 0xc0091964b0})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d6a738?, 0xc0091964b0?})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
	github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2005094
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96

Log preceding fatal error

=== RUN   TestTableNameCollision
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision485861236
    test_log_scope.go:81: use -show-logs to present logs inline
=== RUN   TestTableNameCollision/kafka
    helpers_test.go:862: making server as system tenant
    helpers_test.go:937: making kafka feed factory

Parameters:

TAGS=bazel,gss
stress=true

Help

Same failure on other branches

#128868 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc branch-release-23.2 release-blocker]

_{This test on roachdash | Improve this report!}

Aug 29 '24 06:08 cockroach-teamcity

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2.10-rc @ 456de6170f2bc7af940df456a4470a665624c4b3:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 1989183 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe2db20?)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc00b944800)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d6a738, 0xc001627270})
	github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d6a738, 0xc001627270}, 0x2, 0x3, 0x0, 0x3?, {0x66aa1b4, 0x21}, {0xc002c71c70, 0x1, ...})
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
	github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc01aca96c0, {0x7d6a738?, 0xc001627270})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d6a738?, 0xc001627270?})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
	github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1989121
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96

Log preceding fatal error

=== RUN   TestTableNameCollision
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision1059431209
    test_log_scope.go:81: use -show-logs to present logs inline
=== RUN   TestTableNameCollision/kafka
    helpers_test.go:862: making server as system tenant
    helpers_test.go:937: making kafka feed factory
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection

Parameters:

TAGS=bazel,gss
stress=true

Help

Same failure on other branches

#128868 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc branch-release-23.2 release-blocker]

_{This test on roachdash | Improve this report!}

Aug 30 '24 06:08 cockroach-teamcity

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2.10-rc @ 456de6170f2bc7af940df456a4470a665624c4b3:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2173543 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe2db20?)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc0074d6c00)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
	github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d6a738, 0xc016d14410})
	github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d6a738, 0xc016d14410}, 0x2, 0x3, 0x0, 0x3?, {0x66aa1b4, 0x21}, {0xc011fd9c70, 0x1, ...})
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
	github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
	github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc017133dc0, {0x7d6a738?, 0xc016d14410})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d6a738?, 0xc016d14410?})
	github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
	github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2173374
	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96

Log preceding fatal error

=== RUN   TestTableNameCollision
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision620289502
    test_log_scope.go:81: use -show-logs to present logs inline
=== RUN   TestTableNameCollision/kafka
    helpers_test.go:862: making server as system tenant
    helpers_test.go:937: making kafka feed factory
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection
    testfeed_test.go:303: creating external connection
    testfeed_test.go:306: ran create external connection

Parameters:

TAGS=bazel,gss
stress=true

Help

Same failure on other branches

#128868 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc branch-release-23.2 release-blocker]

_{This test on roachdash | Improve this report!}

Sep 03 '24 06:09 cockroach-teamcity

closing as already fixed.

Nov 21 '24 17:11 asg0451