could not find the requested resource (clusterregistrations.meta.k8s.io CRD)

[adibrastegarnia]

We created a cluster resource, and the bundle gets created, and the fleet-agent gets deployed on the target cluster. However, we see the following error in the fleet-agent when it tries to create the cluster registration CR. We are trying to understand where this clusterregistrations.meta.k8s.io come from? I checked the agent registration code and it uses clusterregistrations.fleet.cattle.io to create the resource.

time="2025-01-27T17:59:35Z" level=warning msg="Cannot find fleet-agent secret, running registration"
time="2025-01-27T17:59:35Z" level=info msg="Creating clusterregistration with id '7xswkt5mx6sp8qb8scgfzvvmdwbh7n67hf8bvtzbzcfzwp894fdvzh' for new token"
time="2025-01-27T17:59:35Z" level=error msg="Failed to register agent: registration failed: cannot create clusterregistration on management cluster for cluster id '7xswkt5mx6sp8qb8scgfzvvmdwbh7n67hf8bvtzbzcfzwp894fdvzh': the server could not find the requested resource (post clusterregistrations.meta.k8s.io)"

[weyfonk]

Which version of Rancher (if any) and Fleet are you using when this happens?

[bernatvadell]

Description

I’m seeing a 404 when the Fleet agent’s init-container tries to register, because it’s attempting to POST to the clusterregistrations.meta.k8s.io endpoint—even though my CRD lives under fleet.cattle.io/v1alpha1. This causes the agent to loop forever, never finding its bootstrap secret.

---

Environment

• Rancher Server: rancher/rancher:v2.11.0

• fleet-agent image: rancher/fleet-agent:v0.11.4

• Charts installed on management cluster (cattle-fleet-system):

  • fleet-crd-105.0.4+up0.11.4 (APP VERSION 0.11.4)
  • fleet-105.0.4+up0.11.4 (APP VERSION 0.11.4)

• Downstream cluster: the embedded k3s in the Rancher Docker container

---

Reproduction Steps

1. Deploy Rancher 2.11.0 with embedded k3s (the “local-node”).

2. Ensure Fleet charts (0.11.4) and CRDs are applied on the management cluster.

3. Watch the fleet-agent init container logs in the cattle-fleet-local-system namespace:

   {"level":"info","ts":"2025-05-08T19:41:25Z","logger":"setup","msg":"starting registration on upstream cluster","namespace":"cattle-fleet-local-system"}
   time="2025-05-08T19:41:25Z" level=warning msg="Cannot find fleet-agent secret, running registration"
   time="2025-05-08T19:41:25Z" level=info msg="Creating clusterregistration with id 'xkvzhnpspptjkszlpbcskxnb56r5nmgwvcdwkwstxc4qs6s4rd8mdw' for new token"
   time="2025-05-08T19:41:25Z" level=error msg="Failed to register agent: registration failed: cannot create clusterregistration on management cluster for cluster id 'xkvzhnpspptjkszlpbcskxnb56r5nmgwvcdwkwstxc4qs6s4rd8mdw': the server could not find the requested resource (post clusterregistrations.meta.k8s.io)"
   

4. Confirm that the CRD is present in the management cluster:

   kubectl get crd | grep clusterregistrations
   # → clusterregistrations.fleet.cattle.io   2024-11-04T14:36:28Z
   

5. Notice the agent is trying to use the meta.k8s.io API-group instead of fleet.cattle.io.

---

Expected Behavior

The agent should POST to:

POST /apis/fleet.cattle.io/v1alpha1/namespaces/<fleet-namespace>/clusterregistrations

…because that is where the ClusterRegistration CRD is defined.

---

Additional Debugging

• The fleet-agent pod is running rancher/fleet-agent:v0.11.4.

• The controller code invokes:

  fc.Fleet().V1alpha1().ClusterRegistration().Create(...)
  

  so it ought to discover the fleet.cattle.io/v1alpha1 group.

---

Question

Why is the agent falling back to clusterregistrations.meta.k8s.io? Is this a bug in the discovery logic for CRDs, or am I missing a step (e.g., missing an APIService or Scheme registration)? Should the group for that resource really be fleet.cattle.io?

Any guidance to get the init-container to register correctly would be much appreciated.

[weyfonk]

We cannot reproduce this. Please reopen if it happens with a more recent Rancher version.