salt
salt copied to clipboard
Published
20 hours ago •
saltstack
saltstack
[TECH DEBT] unknown flags experimental_control_plane and experimental_upload_certs in kubeadm v1.23.0
Description of the tech debt to be addressed, include links and screenshots
I've tried to use kubeadm module to bootstrap k8s cluster (1.23.0) https://docs.saltproject.io/en/latest/ref/modules/all/salt.modules.kubeadm.html#salt.modules.kubeadm.init
experimental_upload_certs and experimental_control_plane are invalid for newer versions of kubeadm
e.g 1.23.0 like in my case.
https://github.com/kubernetes/kubeadm/issues/2366
https://github.com/saltstack/salt/blob/master/salt/modules/kubeadm.py#L1093-L1094 https://github.com/saltstack/salt/blob/master/salt/modules/kubeadm.py#L1261-L1262
Stack trace:
ID: k8s_kubeadm_init
Function: module.run
Result: False
Comment: An exception occurred in this state: Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/salt/state.py", line 2179, in call
ret = self.states[cdata["full"]](
File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 149, in __call__
return self.loader.run(run_func, *args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1201, in run
return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1216, in _run_as
return _func_or_method(*args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1249, in wrapper
return f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/utils/decorators/__init__.py", line 742, in _decorate
return self._call_function(kwargs)
File "/usr/lib/python3/dist-packages/salt/utils/decorators/__init__.py", line 355, in _call_function
return self._function(*args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/states/module.py", line 424, in run
func_ret = _call_function(
File "/usr/lib/python3/dist-packages/salt/states/module.py", line 470, in _call_function
mret = salt.utils.functools.call_function(__salt__[name], *func_args, **func_kwargs)
File "/usr/lib/python3/dist-packages/salt/utils/functools.py", line 150, in call_function
return salt_function(*function_args, **function_kwargs)
File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 149, in __call__
return self.loader.run(run_func, *args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1201, in run
return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1216, in _run_as
return _func_or_method(*args, **kwargs)
File "/usr/lib/python3/dist-packages/salt/modules/kubeadm.py", line 1125, in init
return _cmd(cmd)
File "/usr/lib/python3/dist-packages/salt/modules/kubeadm.py", line 152, in _cmd
raise CommandExecutionError(result["stderr"])
salt.exceptions.CommandExecutionError: unknown flag: --experimental-upload-certs
To see the stack trace of this error execute with --v=5 or higher
Started: 07:52:46.549030
Duration: 261.487 ms
Changes:
----------
Versions Report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
Salt: 3004
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.7.3
docker-py: 4.1.0
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 2.10.1
libgit2: Not Installed
M2Crypto: 0.31.0
Mako: Not Installed
msgpack: 0.6.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: Not Installed
pycryptodome: 3.6.1
pygit2: Not Installed
Python: 3.8.10 (default, Sep 28 2021, 16:10:42)
python-gnupg: 0.4.5
PyYAML: 5.3.1
PyZMQ: 18.1.1
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.2
System Versions:
dist: ubuntu 20.04 focal
locale: utf-8
machine: x86_64
release: 5.4.0-91-generic
system: Linux
version: Ubuntu 20.04 focal
I've created patch to fix this issue, also I've added control_plane_endpoint so this module can handle multiple masters setup, otherwise it was throwing an error like this while kubeadm.join:
Please ensure that:
* The cluster has a stable controlPlaneEndpoint address.
Patch
--- /usr/lib/python3/dist-packages/salt/modules/kubeadm.py 2021-12-15 11:16:01.484875285 +0000
+++ kubeadm.py.new 2021-12-15 11:39:35.509655920 +0000
@@ -980,9 +980,10 @@
apiserver_cert_extra_sans=None,
cert_dir=None,
certificate_key=None,
+ control_plane_endpoint=None,
config=None,
cri_socket=None,
- experimental_upload_certs=False,
+ upload_certs=False,
feature_gates=None,
ignore_preflight_errors=None,
image_repository=None,
@@ -1024,10 +1025,13 @@
config
Path to a kubeadm configuration file
+ control_plane_endpoint
+ Specify a stable IP address or DNS name for the control plane
+
cri_socket
Path to the CRI socket to connect
- experimental_upload_certs
+ upload_certs
Upload control-plane certificate to the kubeadm-certs Secret
feature_gates
@@ -1090,8 +1094,8 @@
"""
cmd = ["kubeadm", "init"]
- if experimental_upload_certs:
- cmd.append("--experimental-upload-certs")
+ if upload_certs:
+ cmd.append("--upload-certs")
if skip_certificate_key_print:
cmd.append("--skip-certificate-key-print")
if skip_token_print:
@@ -1104,6 +1108,7 @@
("cert-dir", cert_dir),
("certificate-key", certificate_key),
("config", config),
+ ("control-plane-endpoint", control_plane_endpoint),
("cri-socket", cri_socket),
("feature-gates", feature_gates),
("ignore-preflight-errors", ignore_preflight_errors),
@@ -1172,7 +1177,7 @@
discovery_token=None,
discovery_token_ca_cert_hash=None,
discovery_token_unsafe_skip_ca_verification=False,
- experimental_control_plane=False,
+ control_plane=False,
ignore_preflight_errors=None,
node_name=None,
skip_phases=None,
@@ -1222,7 +1227,7 @@
For token-based discovery, allow joining without
'discovery-token-ca-cert-hash' pinning
- experimental_control_plane
+ control_plane
Create a new control plane instance on this node
ignore_preflight_errors
@@ -1258,8 +1263,8 @@
cmd.append(api_server_endpoint)
if discovery_token_unsafe_skip_ca_verification:
cmd.append("--discovery-token-unsafe-skip-ca-verification")
- if experimental_control_plane:
- cmd.append("--experimental-control-plane")
+ if control_plane:
+ cmd.append("--control-plane")
parameters = [
("apiserver-advertise-address", apiserver_advertise_address),
Now, I successfully created k8s cluster using saltstack :)
kubeadm.init:
k8s_kubeadm_init:
module.run:
- kubeadm.init:
- apiserver_cert_extra_sans: {{ control_plane_endpoint }}
- control_plane_endpoint: {{ control_plane_endpoint }}
- upload_certs: True
- ignore_preflight_errors: "True"
- token: {{ salt['pillar.get']('k8s:token') }}
- token_ttl: {{ salt['pillar.get']('k8s:token_ttl', '24h') }}
- certificate_key: {{ salt['pillar.get']('k8s:certificate_key') }}
- onlyif:
- test ! -f /etc/kubernetes/kubelet.conf
kubeadm.join:
k8s_kubeadm_join_master:
module.run:
- kubeadm.join:
- api_server_endpoint: "{{ salt['pillar.get']('k8s-kube-vip:fqdn') }}:{{ salt['pillar.get']('k8s-kube-vip:port', '6443') }}"
- apiserver_bind_port: {{ salt['pillar.get']('k8s-kube-vip:port', '6443') }}
- discovery_token_unsafe_skip_ca_verification: True
- ignore_preflight_errors: True
- control_plane: True
- token: {{ salt['pillar.get']('k8s:token') }}
- certificate_key: {{ salt['pillar.get']('k8s:certificate_key') }}
- onlyif:
- test ! -f /etc/kubernetes/kubelet.conf
@daro1337 Thanks for the report. Would you be willing to submit your fix above as a PR?
