Unable to enable metallb add-on

[HorusTheSonOfOsiris]

Summary

While enabling the metallb load balancer add-on on microk8s Internal error

What Should Happen Instead?

Metallb load balancer should be enable with address pools and l2advertisements

Reproduction Steps

1. microk8s enable dns
2. microk8s enable metallb

Introspection Report

Inspecting system Inspecting Certificates Inspecting services Service snap.microk8s.daemon-cluster-agent is running Service snap.microk8s.daemon-containerd is running Service snap.microk8s.daemon-kubelite is running Service snap.microk8s.daemon-k8s-dqlite is running Service snap.microk8s.daemon-apiserver-kicker is running Copy service arguments to the final report tarball Inspecting AppArmor configuration Gathering system information Copy processes list to the final report tarball Copy disk usage information to the final report tarball Copy memory usage information to the final report tarball Copy server uptime to the final report tarball Copy openSSL information to the final report tarball Copy snap list to the final report tarball Copy VM name (or none) to the final report tarball Copy current linux distribution to the final report tarball Copy network configuration to the final report tarball Inspecting kubernetes cluster Inspect kubernetes cluster Inspecting dqlite Inspect dqlite

Are you interested in contributing with a fix?

yes

inspection-report-20221227_212144.tar.gz

I am running microk8s cluster on raspberry pi 4. Master node: 8 GB RAM 128 GB disk Worker node: 2 GB RAM 32 GB disk with attached storage 512 GB

I have tried to run it on master by removing the worker node but still getting the same error.

root@k8s-master1:/home/k8s-master1/metallb# microk8s enable metallb Infer repository core for addon metallb Enabling MetalLB Enter each IP address range delimited by comma (e.g. '10.64.140.43-10.64.140.49,192.168.0.105-192.168.0.111'): 192.168.20.110-192.168.20.130 Applying Metallb manifest customresourcedefinition.apiextensions.k8s.io/addresspools.metallb.io created customresourcedefinition.apiextensions.k8s.io/bfdprofiles.metallb.io created customresourcedefinition.apiextensions.k8s.io/bgpadvertisements.metallb.io created customresourcedefinition.apiextensions.k8s.io/bgppeers.metallb.io created customresourcedefinition.apiextensions.k8s.io/communities.metallb.io created customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io created customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io created namespace/metallb-system created serviceaccount/controller created serviceaccount/speaker created clusterrole.rbac.authorization.k8s.io/metallb-system:controller unchanged clusterrole.rbac.authorization.k8s.io/metallb-system:speaker unchanged role.rbac.authorization.k8s.io/controller created role.rbac.authorization.k8s.io/pod-lister created clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller unchanged clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker unchanged rolebinding.rbac.authorization.k8s.io/controller created secret/webhook-server-cert created service/webhook-service created rolebinding.rbac.authorization.k8s.io/pod-lister created daemonset.apps/speaker created deployment.apps/controller created validatingwebhookconfiguration.admissionregistration.k8s.io/validating-webhook-configuration created Waiting for Metallb controller to be ready.

deployment.apps/controller condition met Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "ipaddresspoolvalidationwebhook.metallb.io": failed to call webhook: Post "https://webhook-service.metallb-system.svc:443/validate-metallb-io-v1beta1-ipaddresspool?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "cert") Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "l2advertisementvalidationwebhook.metallb.io": failed to call webhook: Post "https://webhook-service.metallb-system.svc:443/validate-metallb-io-v1beta1-l2advertisement?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "cert")

[ktsakalozos]

Hi @jayantraizada, what else can you tell me about your setup? I am not able to reproduce this issue.

[HorusTheSonOfOsiris]

Hi @ktsakalozos Sorry for the late response. This was resolved when I switched to snap's edge channel.

[BoKKeR]

I am facing the same problem but I did not change the channel since that would require reinstalling the cluster, what should I provide to be able to debug this issue?

bokker@k8s-nuc:~$ microk8s enable metallb
Infer repository core for addon metallb
Enabling MetalLB
Enter each IP address range delimited by comma (e.g. '10.64.140.43-10.64.140.49,192.168.0.105-192.168.0.111'): 10.0.0.31-10.0.0.40
Applying Metallb manifest
customresourcedefinition.apiextensions.k8s.io/addresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bfdprofiles.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgpadvertisements.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgppeers.metallb.io created
customresourcedefinition.apiextensions.k8s.io/communities.metallb.io created
customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io created
namespace/metallb-system created
serviceaccount/controller created
serviceaccount/speaker created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller unchanged
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker unchanged
role.rbac.authorization.k8s.io/controller created
role.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller unchanged
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker unchanged
rolebinding.rbac.authorization.k8s.io/controller created
secret/webhook-server-cert created
service/webhook-service created
rolebinding.rbac.authorization.k8s.io/pod-lister created
daemonset.apps/speaker created
deployment.apps/controller created
validatingwebhookconfiguration.admissionregistration.k8s.io/validating-webhook-configuration configured
Waiting for Metallb controller to be ready.
deployment.apps/controller condition met
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "ipaddresspoolvalidationwebhook.metallb.io": failed to call webhook: Post "https://webhook-service.metallb-system.svc:443/validate-metallb-io-v1beta1-ipaddresspool?timeout=10s": dial tcp 10.152.183.108:443: connect: connection refused
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "l2advertisementvalidationwebhook.metallb.io": failed to call webhook: Post "https://webhook-service.metallb-system.svc:443/validate-metallb-io-v1beta1-l2advertisement?timeout=10s": dial tcp 10.152.183.108:443: connect: connection refused

OS: Ubuntu 22.04.1 LTS x86_64
Kernel: 5.15.0-58-generic
CPU: Intel Celeron J3455 (4) @ 2.300GHz

Client Version: v1.26.1
Kustomize Version: v4.5.7
Server Version: v1.25.5

I get this error when trying to add a config:

Failed sync attempt to dc8fb54a667b977cd9c3ebdc326dfc806e37e202: one or more objects failed to apply, reason: Internal error occurred: failed calling webhook "addresspoolvalidationwebhook.metallb.io": failed to call webhook: Post "https://webhook-service.metallb-system.svc:443/validate-metallb-io-v1beta1-addresspool?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "cert") (retried 5 times).

EDIT: I tried running: sudo snap refresh microk8s --channel=edge

and it gives the same result

[ubune]

Hi, same problem here. OS: Ubuntu 22.04.1 LTS x86_64 5.15.0-60-generic MicroK8s v1.26.1 revision 4720

I had to delete my cluster and reinstall with edge-channel to avoid issue.

[neoaggelos]

Hi @ubune and everyone else subscribed/affected to this issue:

Fix has been merged into master https://github.com/canonical/microk8s-core-addons/pull/160, and backported to 1.25 and 1.26 branches. It is now available on 1.26/edge and 1.25/edge and will soon be pushed to 1.25/stable and 1.26/stable with the next Kubernetes patchlevel release

[weirlive]

I'm still seeing this same issue:

installed: v1.26.3

Do I need to switch to something else? I installed metallb from the manifest which worked, but I wanted to use the Microk8s version and in doing so have hit the same error that is posted above.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[btbxbob]

i have the same issue in 1.28.7

Error from server (InternalError): error when creating "config.yaml": Internal error occurred: failed calling webhook "ipaddresspoolvalidationwebhook.metallb.io": failed to call webhook: Post "https://webhook-service.metallb-system.svc:443/validate-metallb-io-v1beta1-ipaddresspool?timeout=10s": EOF
Error from server (InternalError): error when creating "config.yaml": Internal error occurred: failed calling webhook "l2advertisementvalidationwebhook.metallb.io": failed to call webhook: Post "https://webhook-service.metallb-system.svc:443/validate-metallb-io-v1beta1-l2advertisement?timeout=10s": EOF