eventing icon indicating copy to clipboard operation
eventing copied to clipboard

Upgrade test for Broker panics in fetchURL

Open pierDipi opened this issue 1 year ago • 1 comments

Describe the bug

https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative-extensions_eventing-kafka-broker/3908/upgrade-tests_eventing-kafka-broker_main/1795832023298871296

        --- FAIL: TestUpgrades/Run/KafkaBrokerContinualTest#01 (16.05s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x2d3c075]

goroutine 3084 [running]:
testing.tRunner.func1.2({0x30d5500, 0x4d83480})
	/root/.gvm/gos/go1.22.3/src/testing/testing.go:1631 +0x3f7
testing.tRunner.func1()
	/root/.gvm/gos/go1.22.3/src/testing/testing.go:1634 +0x6b6
panic({0x30d5500?, 0x4d83480?})
	/root/.gvm/gos/go1.22.3/src/runtime/panic.go:770 +0x132
knative.dev/eventing-kafka-broker/test/upgrade/continual.(*kafkaBrokerSut).fetchURL(0xc001f8f780, {{0x39dd548?, 0x4e31a40?}, 0xc000ce81a8?, 0xc000f16dc0?})
	/home/prow/go/src/knative.dev/eventing-kafka-broker/test/upgrade/continual/broker.go:158 +0x575
knative.dev/eventing-kafka-broker/test/upgrade/continual.kafkaBrokerSut.Deploy({{{0x34664e9, 0x19}, {0x34551fe, 0xf}, 0xc000851b30, 0xc0008852f0}, {{0x3467ed1, 0x1a}, {{0x4d9c7a0, 0x2, ...}}}}, ...)
	/home/prow/go/src/knative.dev/eventing-kafka-broker/test/upgrade/continual/broker.go:100 +0x11f
knative.dev/eventing/test/upgrade/prober.(*prober).deployConfiguration(0xc0026a68a0)
	/home/prow/go/src/knative.dev/eventing-kafka-broker/vendor/knative.dev/eventing/test/upgrade/prober/configuration.go:160 +0x517
knative.dev/eventing/test/upgrade/prober.(*prober).deploy(0xc0026a68a0)
	/home/prow/go/src/knative.dev/eventing-kafka-broker/vendor/knative.dev/eventing/test/upgrade/prober/prober.go:121 +0x132
knative.dev/eventing/test/upgrade/prober.(*probeRunner).Setup(0xc0030ae320, {0xc0012c0820?, 0xc000ce81a8?})
	/home/prow/go/src/knative.dev/eventing-kafka-broker/vendor/knative.dev/eventing/test/upgrade/prober/prober.go:61 +0x132
knative.dev/eventing/test/upgrade/prober.NewContinualVerification.func1({0xc0012c0820?, 0xc000ce81a8?})
	/home/prow/go/src/knative.dev/eventing-kafka-broker/vendor/knative.dev/eventing/test/upgrade/prober/continual.go:53 +0x2ff
knative.dev/pkg/test/upgrade.(*suiteExecution).runContinualTests.func1(0xc0012c0820)
	/home/prow/go/src/knative.dev/eventing-kafka-broker/vendor/knative.dev/pkg/test/upgrade/steps.go:63 +0x142
testing.tRunner(0xc0012c0820, 0xc001694720)
	/root/.gvm/gos/go1.22.3/src/testing/testing.go:1689 +0x21f
created by testing.(*T).Run in goroutine 3495
	/root/.gvm/gos/go1.22.3/src/testing/testing.go:1742 +0x826

Expected behavior

  • No panic but a clear error of what happened

To Reproduce

  • Unclear, it's not reliably reproducible

Knative release version

Additional context

/area test-and-release

pierDipi avatar May 30 '24 08:05 pierDipi

@pierDipi on the relevant line in the error we have:

	url := br.Status.Address.URL

Since Address is a pointer, it stands to reason that this might happen when the address is not set on the broker (and then we try to access the URL)

What I'm not sure about is how this can happen, since before we get the broker we wait to see if the resource is ready. Maybe since it's the continual test we have a race condition like:

Broker ready -> WaitForResourceReady succeeds -> upgrade happens -> broker goes not ready -> get broker -> access addess (and panic)

But, that is only a hunch...

Cali0707 avatar May 30 '24 18:05 Cali0707