ansible-role-rke2 icon indicating copy to clipboard operation
ansible-role-rke2 copied to clipboard

bug: kubelet server certificates does not include keepalived VIP

Open moray95 opened this issue 10 months ago • 11 comments

Summary

When using HA setup with Keeplived, the server certificates provisioned for Kubelet does not include the Keepalived VIP. This causes TLS verification issues when performing various operations like viewing logs or port forwarding on the current leader.

Issue Type

Bug Report

Ansible Version

ansible [core 2.14.6]
  config file = /Users/moray/.ansible.cfg
  configured module search path = ['/Users/moray/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/homebrew/Cellar/ansible/7.6.0/libexec/lib/python3.11/site-packages/ansible
  ansible collection location = /Users/moray/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/homebrew/bin/ansible
  python version = 3.11.4 (main, Jul 25 2023, 17:36:13) [Clang 14.0.3 (clang-1403.0.22.14.1)] (/opt/homebrew/Cellar/ansible/7.6.0/libexec/bin/python3.11)
  jinja version = 3.1.2
  libyaml = True

Steps to Reproduce

  1. Install RKE2 (sample playbook below)
- hosts: rke
  become: true
  roles:
    - role: lablabs.rke2
  vars:
    rke2_ha_mode: true
    rke2_ha_mode_keepalived: true
    rke2_version: v1.26.7+rke2r1
    rke2_install_bash_url: https://get.rke2.io
    rke2_api_ip: 10.64.0.9
    rke2_disable:
      - rke2-ingress-nginx
    rke2_cni: canal
    rke2_cluster_group_name: rke
    rke2_servers_group_name: rke_master
    # Ansible group including worker nodes
    rke2_agents_group_name: rke_worker
    rke2_server_options:
      - "disable-cloud-controller: true"
  1. Try viewing logs of any pod on the current Keepalived leader

Expected Results

The TLS certificate generated for Kubelet includes the Keepalived VIP (10.64.0.9 in the example above), issuing kubectl logs and kubectl port-forward command on pods on the current leader works without problem.

Actual Results

The TLS certificate for Kubelet does not include the Keepalived VIP (10.64.0.9 in the example above). Issuing kubectl logs or kubectl port-forward commands on pods on the current leader results in the following error:

Error from server: Get "https://10.64.0.9:10250/containerLogs/kube-system/kube-proxy-master-0/kube-proxy": tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, 10.64.0.10, not 10.64.0.9

Additional information:

  • The API server serving certificate does include the VIP.
  • The leader's internal ip address always shows up as the VIP.
  • I have tried setting RKE2 options node-ip and advertise-address to the non-virtual ip but to no avail.

moray95 avatar Aug 28 '23 13:08 moray95