ansible-rpi-k8s-cluster icon indicating copy to clipboard operation
ansible-rpi-k8s-cluster copied to clipboard

Static Routing issue (Kali VM) and kube-apiserver process using 100%CPU

Open thesecmaestro opened this issue 6 years ago • 0 comments

Configuration: Ansible Control Machine (Kali 2018.4 VM), using VMWare Workstation (natt'ed network) Pull of ansible-rpi-k8s-project of 11/7/18.

Since both issues described here (static routing issue) and CPU spike kube-apiserver might be related keeping issues together in one report for now.

Problem 1: Static Routing issue - SSH Kali host Cannot ssh from Ansible Control Machine (192.168.80.135) to rpi-k8s-1 (192.168.1.100, statically assigned). ssh to WIFI (DHCP) address (15.15.15.147) works fine

Expected results: ssh from 192.168.80.135 to 192.168.1.100 to work (Similar as to 15.15.15.147)

Problem 2: kube-apiserver process consumes CPU and freezes rpi-k8s-1 within minutes After successful execution of ansible-playbook -i inventory playbooks/deploy.yml, kube-apiserver (by resolving issue 1 & 2 listed below) process runs out of memory and freezes rpi-k8s-1 within minutes Workaround: manually kill kube-apiserver process each few minutes

Expected results: Expect rpi-k8s-1 to continue to function (cap memory usage of of kube-apiserver process)

Resolved Issue 1: ssh to 15.15.15.147 (WIFI rpi-k8s-1) fails. SOLVED: Updated ansible.cfg: ssh_args = -o IPQos=throughput

Resolved Issue 2: ansible-playbook -i inventory playbooks/deploy.yml fails to contact slave nodes

Snippet error in output

…..
TASK [Gathering Facts] ********************************************************************************************
Tuesday 06 November 2018  13:03:26 -0800 (0:00:01.136)       0:00:50.157 ******
fatal: [rpi-k8s-2]: UNREACHABLE! => {"changed": false, "msg": "SSH Error: data could not be sent to remote host \"192.168.100.128\". Make sure this host can be reached over ssh", "unreachable": true}
fatal: [rpi-k8s-3]: UNREACHABLE! => {"changed": false, "msg": "SSH Error: data could not be sent to remote host \"192.168.100.129\". Make sure this host can be reached over ssh", "unreachable": true}

SOLVED: Updated inventory/group_vars/rpi_k8s_slaves/all.yml: ansible_ssh_common_args: '-o ProxyCommand="ssh -o IPQos=throughput -W %h:%p -q {{ ansible_user }}@{{ jumphost_ip }}"'

After resolving issue 1 & 2 : ansible-playbook -i inventory playbooks/deploy.yml completes successfully

Continued/existing problems:

Problem 1: Cannot ssh to rpi-k8s-1 (192.168.1.100, statically assigned) Ssh to rpi-k8s-1 (15.15.15.147, wifi, dhcp) works fine

Problem 2: Kube-apiserver consumes 100%cpu every few minutes and freezes rpi-k8s-1

thesecmaestro avatar Nov 08 '18 17:11 thesecmaestro