F5Networks
multicluster CIS with namespace-label is not working correctly
Setup Details
CIS Version : 2.17.1
Build: f5networks/k8s-bigip-ctlr:latest
BIGIP Version: Big IP v16.1.3.1
AS3 Version: 3.47
Agent Mode: AS3
Orchestration: K8S
Orchestration Version: v1.28.10
Pool Mode: Nodeport
Description
In multicluster CIS deployed in active-active mode with namespace-label defined in values files, we noticed that the Transport server only contains the pool member of the primary cluster. Removing the namespace-label works with out any issue.
Steps To Reproduce
- Deploy CIS in active-active mode
- Values file should include a namespace label
- Create namespace with labels on both clusters
- Try to create a Transport server and check for pool members.
Expected Result
The pool should have members from both clusters.
Actual Result
Pool members only have members from the primary cluster
values file: log_level: DEBUG namespace_label: "f5cis-enable=true" pool_member_type: auto insecure: true custom-resource-mode: true log-as3-response: true ipam : false multi-cluster-mode: primary extended-spec-configmap: f5-cis/global-spec-config as3-validation: true
Examples using namespace labels will also help the user community.
Also when namespace label is used primary and secondary CIS entering into split brain and posting declaration independently overriding each other.
@arzzon I have tested https://github.com/F5Networks/k8s-bigip-ctlr/pull/3557 and It didn´t fix it with my configuration. Tested with image quay.io/f5networks/k8s-bigip-ctlr-devel:2.18.1-26-Sept provided by Vivek. Please see next a test with 3 clusters (ocp1, ocp2 and ocp3)
-
When using the CIS config cis-config/f5bigipctlr.ocp1.yaml.works (not speciying namespace or namespace-label parameters, the log logs/cis-all-namespaces.log shows no warnings and pool members from 3 clusters are discovered.
-
When using the CIS config cis-config/f5bigipctlr.ocp1.yaml.fails (specifying namespace-label), the log logs/cis-namespace-label.log shows the following warnings and the pool members from ocp3 (external) are not discovered:
% egrep -i "(error|warning)" logs/cis-namespace-label.log
2024/09/26 11:08:22 [WARNING] Creating GTM with default bigip credentials as GTM BIGIP Url or GTM BIGIP Username or GTM BIGIP Password is missing on CIS args.
2024/09/26 11:08:26 [WARNING] Ensure Global Extended Configmap is created in CIS monitored namespace
2024/09/26 11:08:26 [WARNING] [MultiCluster] informer not found for namespace: f5bigipctlr while fetching secret for cluster ocp2
2024/09/26 11:08:29 [WARNING] [MultiCluster] informer not found for namespace: f5bigipctlr while fetching secret for cluster ocp3
2024/09/26 11:08:37 [ERROR] Pool Members could not be fetched for service {cigna-route-a ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:37 [ERROR] Pool Members could not be fetched for service {cigna-route-b ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:37 [ERROR] Pool Members could not be fetched for service {cigna-route-a ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:37 [ERROR] Pool Members could not be fetched for service {cigna-route-b ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:40 [ERROR] Pool Members could not be fetched for service {cigna-route-a ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:40 [ERROR] Pool Members could not be fetched for service {cigna-route-b ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:40 [ERROR] Pool Members could not be fetched for service {cigna-route-a ocp3 openshift-ingress} with targetPort 0:443
2024/09/26 11:08:40 [ERROR] Pool Members could not be fetched for service {cigna-route-b ocp3 openshift-ingress} with targetPort 0:443
Note that the pool members for ocp2 are populated yet the following are shown:
2024/09/26 11:08:26 [WARNING] [MultiCluster] informer not found for namespace: f5bigipctlr while fetching secret for cluster ocp2
2024/09/26 11:08:29 [WARNING] [MultiCluster] informer not found for namespace: f5bigipctlr while fetching secret for cluster ocp3
