Establish ownership resource relationship (e.g. Kafka and KafkaTopic)

[jonatasbaldin]

Hi there 👋

We should map and implement ownership relationship between dependent resources (probably using the SetControllerReference.

For example, when removing a Kafka resource, the KafkaTopic is left hanging around, even though it doesn't exist anymore.

We should add a controller reference when creating these child resources, like:

• PG -> Database, ConnectionPool, ServiceUser
• Kafka -> KafkaTopic, KafkaACL, KafkaConnect, KafkaSchema, ServiceUser

What do you think?

[mhoffm-aiven]

Hey @jonatasbaldin,

There are some issues with the ownership approach. For example if you have a kafka resource in a different namespace than the kafka topic ( i can imagine this to be a common usecase since a kafka deployment is kind of fat and should be shared between applications ) you cannot add ownership references to resources in different namespaces.

For situations where the owner is missing ( as it is a previously existing resource that was created via TF or Console ) we should not error so this should situation should be regarded as expected.

What i want to say: This approach will sometimes work and sometimes not and is probably confusing. Sometimes the topic is just deleted, othertimes it exists even though the kafka was deleted ( different namespace ). As such i would propose not going for it at all. We can then add a DELETED_AT_AIVEN state maybe and observe that during reconciliation ( when we requeue even after successful creation ).

What do you think?

I have implemented it though: https://github.com/aiven/aiven-kubernetes-operator/pull/140

[mhoffm-aiven]

One more issue is that some resource have just general "Services" as parents ( think service user or service integrations ). We can then only guess which service it is. There is the possibility that two services with the same name exist ( one cannot be created probably by API constraints ) but we would be able to add the wrong service as owner.