cluster-api-provider-vcluster

cluster-api-provider-vcluster copied to clipboard

What happened?

Creating a cluster with kubernetes version 1.26 results in cluster version 1.24.

Input

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: VCluster
metadata:
  name: student01
  namespace: student01-vcluster
spec:
  controlPlaneEndpoint:
    host: ""
    port: 0
  helmRelease:
    chart:
      name: null
      repo: null
      version: null
    values: |
      sync:
        ingresses:
          enabled: true
        networkpolicies:
          enabled: false
        #for metricsserver
        nodes:
          enabled: true
          syncAllNodes: true
        #makes SCs visible
        # hoststorageclasses:
        #   enabled: true
        persistentvolumes:
          enabled: true
      syncer:
        extraArgs:
        - --tls-san=student01.student01-vcluster
  kubernetesVersion: 1.26.0

Log snippet

1.6793202193629904e+09	DEBUG	vcluster-controller	upgrade virtual cluster helm chart student01-vcluster/student01
1.679320219363043e+09	INFO	vcluster-controller	vclusters student01-vcluster/student01 patch version defined in .spec.kubernetesVersion field will be ignored, latest supported patch version will be used
1.6793202193633888e+09	INFO	vcluster-controller	officially unsupported host server version 1.26, will fallback to virtual cluster version v1.24
1.6793202193639154e+09	INFO	vcluster-controller	Deploy virtual cluster student01-vcluster/student01 with values: sync:
  ingresses:
    enabled: true
  networkpolicies:
    enabled: false
  nodes:
    enabled: true
    syncAllNodes: true
  persistentvolumes:
    enabled: true
syncer:
  extraArgs:
  - --tls-san=student01.t9v.de
  - --tls-san=student01.student01-vcluster
vcluster:
  image: rancher/k3s:v1.24.1-k3s1

helm upgrade student01 vcluster --repo https://charts.loft.sh --version 0.11.1 --kubeconfig /tmp/62755493 --namespace student01-vcluster --install --values /tmp/4202844644

What did you expect to happen?

vcluster uses 1.26

How can we reproduce it (as minimally and precisely as possible)?

Create a cluster with version 1.26

Anything else we need to know?

The cluster was using vcluster 0.13.1 and was upgraded later.

clusterctl version clusterctl version: &version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.5", GitCommit:"58770484dee6c99c10e32c06652e9f9643f78e9e", GitTreeState:"clean", BuildDate:"2023-03-02T15:57:13Z", GoVersion:"go1.19.6", Compiler:"gc", Platform:"linux/amd64"}

The vcluster controller pod uses the following images

docker.io/loftsh/cluster-api-provider-vcluster:0.1.3 gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0

There is no newer image on dockerhub https://hub.docker.com/r/loftsh/cluster-api-provider-vcluster/tags

Host cluster Kubernetes version

$ kubectl version
kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.1", GitCommit:"8f94681cd294aa8cfd3407b8191f6c70214973a4", GitTreeState:"clean", BuildDate:"2023-01-18T15:58:16Z", GoVersion:"go1.19.5", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.3", GitCommit:"9e644106593f3f4aa98f8a84b23db5fa378900bd", GitTreeState:"clean", BuildDate:"2023-03-15T13:33:12Z", GoVersion:"go1.19.7", Compiler:"gc", Platform:"linux/amd64"}

Host cluster Kubernetes distribution

vanilla v1.26.3, Ubuntu 22.04.2 LTS   5.15.0-67-generic   containerd://1.6.18

vlcuster version

$ vcluster --version
vcluster version 0.14.2

Vcluster Kubernetes distribution(k3s(default)), k8s, k0s)

default (k3s)

OS and Arch

OS: Ubuntu
Arch: AMD64

Mar 20 '23 15:03 everflux

Thank you for reporting this. I am going to update the provider and release a new version soon.

P.S: this should have been reported in the CAPI provider repo - loft-sh/cluster-api-provider-vcluster, but we can leave it here this time.

Mar 22 '23 17:03 matskiv

@matskiv is something blocking the release, anything that can be contributed?

Oct 31 '23 09:10 everflux

I didn't get to release this due to other priorities at the time, and right now I don't work on this project so I don't have any updates I can provide.

Oct 31 '23 10:10 matskiv

@everflux Could you test using the latest release v0.2.0-alpha.1 to see if this is fixed?

May 02 '24 22:05 deniseschannon

@deniseschannon How do I test the release v0.2.0-alpha.1? I tried executing following commands, but I get the errors shown below.

clusterctl init --infrastructure=vcluster:v0.2.0-alpha.1 Fetching providers Error: invalid provider metadata: version v0.2.0-alpha.1 for the provider cluster-api-provider-vcluster-system/infrastructure-vcluster does not match any release series

clusterctl init --infrastructure=vcluster:v0.2.0 Fetching providers Error: failed to get provider components for the "vcluster:v0.2.0" provider: failed to read "infrastructure-components.yaml" from provider's repository "infrastructure-vcluster": release not found for version v0.2.0, please retry later or set "GOPROXY=off" to get the current stable release: 404 Not Found

Jun 07 '24 12:06 nathanielkeeler

clusterctl init --infrastructure=vcluster:v0.2.0-alpha.1

@nathanielkeeler Your first command clusterctl init --infrastructure=vcluster:v0.2.0-alpha.1 was absolutely correct. The current minor version bump to 0.2 was not reflected in the attached metadata.yaml file. So clusterctl could not find a suiting match for the given version. Can you please try again.

UPDATE: We recently released v0.2.0-alpha.2, which contains an important bug fix. Feel free to use that.

Jun 12 '24 15:06 johannesfrey

With v0.2.0-beta.1, we have removed the ability to set Kubernetes version in CAPI directly, you should update k8s version with a vcluster.yaml in the latest vCluster release v0.20.0-beta.15.

Since we have independent distros within vcluster, it makes more sense to configure it at the vcluster itself instead of using CAPI methods.

Aug 09 '24 13:08 deniseschannon