kubermatic
WIP: Set `floating-network-id` in OpenStack cloud-config to provided floating IP pool
What this PR does / why we need it: This hopefully addresses #12905 - seems that we have missed setting an explicit floating network in the cloud-config passed to the CCM. That is fine when your OpenStack setup only has a single external network (like the one we are testing against), but becomes problematic if there are multiple.
It seems to be sensible to default to the same external network that is used for node floating IPs because we know that the router connection has to be there. If people want to use other networks, they have the option to set the loadbalancer.openstack.org/floating-network-id annotation on their Service objects.
Which issue(s) this PR fixes:
Fixes #12905
What type of PR is this? /kind bug
Special notes for your reviewer:
Does this PR introduce a user-facing change? Then add your Release Note here:
Explicitly configure OpenStack CCM with floating IP pool configured for user cluster instead of defaulting to first external network available
Documentation:
NONE
/hold
{"level":"info","time":"2024-01-18T14:51:22.357Z","caller":"tests/loadbalancer.go:57","msg":"Provider does not support LoadBalancers, skipping.","scenario":"openstack-ubuntu-1.28.2","cluster":"kkp-1747982789875077120-25rjg"}
Looks like LBs aren't tested in e2e, I need to investigate why.
@xrstf: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:
-
/test pre-kubermatic-ccm-migration-aws-e2e -
/test pre-kubermatic-ccm-migration-azure-e2e -
/test pre-kubermatic-ccm-migration-gcp-e2e -
/test pre-kubermatic-cilium-e2e -
/test pre-kubermatic-dualstack-e2e-aws-canal -
/test pre-kubermatic-dualstack-e2e-aws-cilium -
/test pre-kubermatic-e2e-anexia-flatcar-1.29 -
/test pre-kubermatic-e2e-aws-ubuntu-1.26 -
/test pre-kubermatic-e2e-aws-ubuntu-1.27 -
/test pre-kubermatic-e2e-aws-ubuntu-1.28 -
/test pre-kubermatic-e2e-aws-ubuntu-1.28-ce -
/test pre-kubermatic-e2e-aws-ubuntu-1.29 -
/test pre-kubermatic-e2e-azure-ubuntu-1.29 -
/test pre-kubermatic-e2e-do-centos-1.29 -
/test pre-kubermatic-e2e-gcp-offline -
/test pre-kubermatic-e2e-gcp-ubuntu-1.26 -
/test pre-kubermatic-e2e-gcp-ubuntu-1.27 -
/test pre-kubermatic-e2e-gcp-ubuntu-1.28 -
/test pre-kubermatic-e2e-gcp-ubuntu-1.29 -
/test pre-kubermatic-e2e-hetzner-ubuntu-1.29 -
/test pre-kubermatic-e2e-kubevirt-centos-1.29 -
/test pre-kubermatic-e2e-kubevirt-ubuntu-1.29 -
/test pre-kubermatic-e2e-nutanix-centos-1.29 -
/test pre-kubermatic-e2e-nutanix-ubuntu-1.29 -
/test pre-kubermatic-e2e-openstack-centos-1.29 -
/test pre-kubermatic-e2e-openstack-ubuntu-1.29 -
/test pre-kubermatic-e2e-packet-ubuntu-1.29 -
/test pre-kubermatic-e2e-vmware-cloud-director-ubuntu-1.29 -
/test pre-kubermatic-e2e-vsphere-ubuntu-1.29 -
/test pre-kubermatic-e2e-vsphere-ubuntu-1.29-customfolder -
/test pre-kubermatic-e2e-vsphere-ubuntu-1.29-datastore-cluster -
/test pre-kubermatic-etcd-launcher-e2e -
/test pre-kubermatic-expose-strategy-e2e -
/test pre-kubermatic-ipam-e2e -
/test pre-kubermatic-konnectivity-e2e -
/test pre-kubermatic-kubermatic-validate-prow-yaml -
/test pre-kubermatic-legacy-machine-controller-user-data-e2e -
/test pre-kubermatic-lint -
/test pre-kubermatic-mla-e2e -
/test pre-kubermatic-nodeport-proxy-e2e -
/test pre-kubermatic-opa-e2e -
/test pre-kubermatic-shellcheck -
/test pre-kubermatic-simulate-github-release -
/test pre-kubermatic-test -
/test pre-kubermatic-test-helm-charts -
/test pre-kubermatic-test-integration -
/test pre-kubermatic-test-user-ssh-key-agent-multiarch -
/test pre-kubermatic-verify
The following commands are available to trigger optional jobs:
-
/test pre-kubermatic-dualstack-e2e-alibaba -
/test pre-kubermatic-dualstack-e2e-azure-canal -
/test pre-kubermatic-dualstack-e2e-azure-cilium -
/test pre-kubermatic-dualstack-e2e-canal -
/test pre-kubermatic-dualstack-e2e-cilium -
/test pre-kubermatic-dualstack-e2e-digitalocean -
/test pre-kubermatic-dualstack-e2e-equinix-centos -
/test pre-kubermatic-dualstack-e2e-equinix-flatcar -
/test pre-kubermatic-dualstack-e2e-equinix-rockylinux -
/test pre-kubermatic-dualstack-e2e-equinix-ubuntu -
/test pre-kubermatic-dualstack-e2e-gcp -
/test pre-kubermatic-dualstack-e2e-hetzner -
/test pre-kubermatic-dualstack-e2e-openstack-canal -
/test pre-kubermatic-dualstack-e2e-openstack-cilium -
/test pre-kubermatic-dualstack-e2e-vsphere -
/test pre-kubermatic-e2e-vsphere-ubuntu-1.29-basepath -
/test pre-kubermatic-trivy-scan
Use /test all to run the following jobs that were automatically triggered:
-
pre-kubermatic-e2e-aws-ubuntu-1.26 -
pre-kubermatic-e2e-aws-ubuntu-1.27 -
pre-kubermatic-e2e-aws-ubuntu-1.28 -
pre-kubermatic-e2e-aws-ubuntu-1.28-ce -
pre-kubermatic-e2e-aws-ubuntu-1.29 -
pre-kubermatic-etcd-launcher-e2e -
pre-kubermatic-expose-strategy-e2e -
pre-kubermatic-kubermatic-validate-prow-yaml -
pre-kubermatic-lint -
pre-kubermatic-mla-e2e -
pre-kubermatic-nodeport-proxy-e2e -
pre-kubermatic-opa-e2e -
pre-kubermatic-test -
pre-kubermatic-test-integration -
pre-kubermatic-verify
In response to this:
/hold cancel /test pre-kubermatic-e2e-openstack-ubuntu-1.28
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
I0118 19:11:43.759351 1 event.go:307] "Event occurred" object="lb-test-1/test" fieldPath="" kind="Service" apiVersion="v1" type="Warning" reason="SyncLoadBalancerFailed" message="Error syncing load balancer: failed to ensure load balancer: error creating LB floatingip: Bad request with: [POST https://api.dbl.cloud.syseleven.net:9696/v2.0/floatingips], error message: {"NeutronError": {"message": "Invalid input for floating_network_id. Reason: 'ext-net' is not a valid UUID.", "type": "HTTPBadRequest", "detail": ""}}"
In case anyone doesn't want to hunt for the error message.
I0118 19:11:43.759351 1 event.go:307] "Event occurred" object="lb-test-1/test" fieldPath="" kind="Service" apiVersion="v1" type="Warning" reason="SyncLoadBalancerFailed" message="Error syncing load balancer: failed to ensure load balancer: error creating LB floatingip: Bad request with: [POST https://api.dbl.cloud.syseleven.net:9696/v2.0/floatingips], error message: {"NeutronError": {"message": "Invalid input for floating_network_id. Reason: 'ext-net' is not a valid UUID.", "type": "HTTPBadRequest", "detail": ""}}"
In case anyone doesn't want to hunt for the error message.
Yes, I need to translate network name to network ID.
/cc @xrstf @mohamed-rafraf
@mohamed-rafraf if you are interested in reviewing a PR!
LGTM label has been added.
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: xrstf
The full list of commands accepted by this bot can be found here.
The pull request process is described here
- ~~pkg/provider/cloud/OWNERS~~ [xrstf]
- ~~pkg/resources/OWNERS~~ [xrstf]
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
@embik unfortunately this didn't fix the issue stated in https://github.com/kubermatic/kubermatic/issues/12905
Hi @bassemtadros can you please show us the cloud-config and Cluster object (with sensitive information redacted) in #12905 so we understand how your Clusters look like in KKP 2.26 (this PR is only available in 2.26.0 and higher)?
Hi @embik ,you can see below in the preset the external network is named external, while in the cluster object it annotates the floating ip pool id of the the other external network, and in the cluster object floatingIPPool it gets its name manila-network, let me know if you need further updates.
kind: Preset
metadata:
creationTimestamp: "2024-10-28T07:27:04Z"
generation: 1
name: datapro
resourceVersion: "49451354"
uid: 48bd8505-cf0a-4fd0-9993-eeb42f77ac0c
spec:
enabled: true
openstack:
datacenter: eum-cf-waw3-1
domain: xxxxx
floatingIPPool: external
network: kkp-k8s-private
routerID: xxxxx-537dc98e4182
subnetID: xxxx-10091aab018a
projects:
- v10fnv6f2y
kind: Cluster
metadata:
annotations:
kubermatic.k8c.io/openstack-floating-ip-pool-id: xxxxxxx-5deb5813602e
presetName: datapro
creationTimestamp: "2024-10-28T07:28:17Z"
finalizers:
- kubermatic.k8c.io/cleanup-credentials-secrets
- kubermatic.k8c.io/cleanup-etcdbackupconfigs
- kubermatic.k8c.io/cleanup-kubermatic-constraints
- kubermatic.k8c.io/cleanup-namespace
- kubermatic.k8c.io/cleanup-openstack-security-group
- kubermatic.k8c.io/cleanup-usersshkeys-cluster-ids
- kubermatic.k8c.io/delete-nodes
generation: 5
labels:
argocd.argoproj.io/instance: kubermatic
env: val
is-credential-preset: "true"
project-id: v10fnv6f2y
name: av6clvgml4
resourceVersion: "49486609"
uid: a8a146bb-c5ca-40f0-bf3f-a2b4e4e16329
spec:
auditLogging: {}
cloud:
dc: eum-cf-waw3-1
openstack:
credentialsReference:
name: credential-openstack-av6clvgml4
namespace: kubermatic
floatingIPPool: manila-network
network: kkp-k8s-private
nodePortsAllowedIPRanges:
cidrBlocks: []
routerID: xxxxx-8b96-537dc98e4182
securityGroups: kubernetes-av6clvgml4
subnetID: xxxxxxx-10091aab018a
providerName: openstack
Thanks for reporting this @bassemtadros and apologies for the issues caused. #13834 will fix this regression, it will be backported to 2.26 and should be included in an upcoming 2.26.1 patch release.