Undefined array key "object"

[daveneeley]

Hi there! I'm super grateful for this project. I'm using it with pfsense at home.

I'm sure I have something configured wrong, but not being a super pro with pfsense and networking I made some inferences from the docs and they seem to have led to these errors. They flood the logs many times per second. Do you have any tips on my configuration?  

PHP Warning: Undefined array key "object" in phar:///usr/local/bin/kubernetes-pfsense-controller/vendor/travisghansen/kubernetes-client-php/src/KubernetesClient/Watch.php on line 479
PHP Warning: Trying to access array offset on value of type null in phar:///usr/local/bin/kubernetes-pfsense-controller/vendor/travisghansen/kubernetes-client-php/src/KubernetesClient/Watch.php on line 479
PHP Warning: Undefined array key "type" in phar:///usr/local/bin/kubernetes-pfsense-controller/vendor/travisghansen/kubernetes-client-php/src/KubernetesClient/Watch.php on line 611

apiVersion: v1
data:
  config: |
    controller-id: mypfsense
    enabled: true
    plugins:
      haproxy-declarative:
        enabled: true
      pfsense-dns-ingresses:
        defaultEnabled: true
        dnsBackends:
          dnsmasq:
            enabled: false
          unbound:
            enabled: true
        enabled: true
        ingressFieldSelector: cluster.app/hostname
        ingressLabelSelector: null
      pfsense-dns-services:
        dnsBackends:
          dnsmasq:
            enabled: false
          unbound:
            enabled: true
        enabled: true
        serviceFieldSelector: cluster.app/hostname
        serviceLabelSelector: null
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: pfsense
    meta.helm.sh/release-namespace: kpc
  labels:
    app.kubernetes.io/instance: pfsense
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-pfsense-controller
    app.kubernetes.io/version: 0.0.1
    helm.sh/chart: kubernetes-pfsense-controller-0.1.2
  name: pfsense-kubernetes-pfsense-controller-config
  namespace: kpc

[travisghansen]

Welcome! I probably need you to send a sample of a svc and ing to know what you really want to achieve. I do believe your field selectors are invalid but want to see samples so I can properly advise further.

[daveneeley]

cluster.app/hostname: grafana on the ingress creates a record in the "Host Overrides" section of the pfsense "DNS Resolver" (aka unbound) with host=grafana. The IP of the host record is the virtual IP returned by kube-vip that was separately configured in pfsense. Everything works and seems to do what I would expect.

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/name: grafana
  name: victoria-metrics-grafana
  namespace: vmetrics
spec:
  clusterIP: 10.43.144.7
  clusterIPs:
  - 10.43.144.7
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: service
    port: 80
    protocol: TCP
    targetPort: 3000
  selector:
    app.kubernetes.io/instance: victoria-metrics
    app.kubernetes.io/name: grafana
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cluster.app/hostname: grafana
  labels:
    app.kubernetes.io/instance: victoria-metrics
    app.kubernetes.io/name: grafana
  name: victoria-metrics-grafana
  namespace: vmetrics
spec:
  ingressClassName: nginx
  rules:
  - host: grafana.my.tld
    http:
      paths:
      - backend:
          service:
            name: victoria-metrics-grafana
            port:
              number: 80
        path: /
        pathType: Prefix
status:
  loadBalancer:
    ingress:
    - ip: 192.168.1.240

[travisghansen]

Yeah those field selectors are not doing what you think they are doing and should be removed from the config. Let's start with that and see if some of the errors go away.

[daveneeley]

I first tried removing the label selectors because they were set to null, but that did not reduce the errors.

I then removed serviceFieldSelector and ingressFieldSelector and set cluster.app/hostname on serviceLabelSelector and ingressLabelSelector instead. This stopped the error logs. I'm curious though - because it seems that all ingresses work whether cluster.app/hostname is set in an annotation or in a label on the ingress. I've deleted entries in pfsense for both types and they all get recreated.