Reconcile all secret with label konghq.com/credential

[piotrwielgolaski-tomtom]

Is there an existing issue for this?

• [X] I have searched the existing issues

Problem Statement

When configuration is build controller ignores all secret until it find reference in any consumer which leads to invalid/incomplete configuration (it must wait for another reconcile cycle - default 2min)

Proposed Solution

• Allow all secret that have label konghq.com/credential in https://github.com/Kong/kubernetes-ingress-controller/blob/e154f202fbb86a9c79f1d17f1e53122d140f228a/internal/controllers/configuration/secret_controller.go#L90

Additional information

This will populate store with all secret that may be used by consumers

Idea in code

func (r *CoreV1SecretReconciler) shouldReconcileSecret(obj client.Object) bool {
	secret, ok := obj.(*corev1.Secret)
	if !ok {
		return false
	}

	labels := secret.Labels
	if labels != nil && labels[CACertLabelKey] == "true" {
		return true
	}

	if _, ok = labels["konghq.com/credential"]; ok {
		return true
	}

	referred, err := r.ReferenceIndexers.ObjectReferred(secret)
	if err != nil {
		r.Log.Error(err, "Failed to check whether secret referred",
			"namespace", secret.Namespace, "name", secret.Name)
		return false
	}

	return referred
}

Acceptance Criteria

No response

[piotrwielgolaski-tomtom]

it should help also for https://github.com/Kong/kubernetes-ingress-controller/issues/5175

[pmalek]

Hi @piotrwielgolaski-tomtom,

Thanks for reporting this issue. #5816 should fix this.

[rainest]

@piotrwielgolaski-tomtom do you know what's actually causing the reconcile by reference to fail and require a second pass? Offhand I'd expect that approach should work, since we shouldn't need Secrets with no references.

tl;dr from https://github.com/Kong/kubernetes-ingress-controller/pull/5816#pullrequestreview-1981025973 do you observe this specifically when creating a KongConsumer and its Secret at the same time (so we aren't necessarily aware of the reference at Secret creation time), or does it happen consistently regardless?

I think we'll probably just go with the simpler approach of ingesting all credentials unless we find we need tighter bound, but it'd be useful to understand the exact nature of this problem should we need to re-introduce reference-based ingest in the future.

[pmalek]

#5819 was created to track the enhancement of not storing all the Secrets with the credential label.