java icon indicating copy to clipboard operation
java copied to clipboard
verified publisher icon kubernetes-client

Unable create Custom Resource Definition with ApiextensionsV1Api.createCustomResourceDefinition()

Open ardhendusaha opened this issue 5 months ago • 6 comments

Describe the bug Hello,

I'm trying to create a custom resource following this official example: https://github.com/kubernetes-client/java/blob/v24.0.0/kubernetes/docs/ApiextensionsV1Api.md#createCustomResourceDefinition

        V1CustomResourceDefinition result = apiInstance.createCustomResourceDefinition(body)
                .dryRun(dryRun)
                .execute();

No matter what I specify in the body , the result is always an HTTP 422 error response from the server:

   {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"CustomResourceDefinition.apiextensions.k8s.io \"widgets.example.com\" is invalid: spec.validation.openAPIV3Schema.dependencies: Forbidden: dependencies is not supported","reason":"Invalid","details":{"name":"widgets.example.com","group":"apiextensions.k8s.io","kind":"CustomResourceDefinition","causes":[{"reason":"FieldValueForbidden","message":"Forbidden: dependencies is not supported","field":"spec.validation.openAPIV3Schema.dependencies"}]},"code":422}

Client Version 24.0.0

Kubernetes Version v1.28.0

Java Version Java 17

To Reproduce Steps to reproduce the behavior:

Copy below java code and execute from local system.

       String kubeConfigPath = "C:\\Users\\Administrator\\WORK\\Projects\\java-client\\kube\\config";
        ApiClient client = Config.fromConfig(kubeConfigPath);
        Configuration.setDefaultApiClient(client);
        ApiextensionsV1Api apiInstance = new ApiextensionsV1Api(client);
        V1CustomResourceDefinition body = new V1CustomResourceDefinition();
        body.setApiVersion("apiextensions.k8s.io/v1");
        body.setKind("CustomResourceDefinition");
        body.setMetadata(new V1ObjectMeta().name("widgets.example.com"));
        List<V1CustomResourceDefinitionVersion> versions = new ArrayList<>();
        V1CustomResourceValidation schema = new V1CustomResourceValidation().openAPIV3Schema(new V1JSONSchemaProps().type("object").dependencies(new HashMap<>()));
        V1CustomResourceDefinitionVersion v1 = new V1CustomResourceDefinitionVersion().name("v1").served(true).storage(true).served(true).schema(schema);
        versions.add(v1);
        body.setSpec(new V1CustomResourceDefinitionSpec().group("example.com").names(new V1CustomResourceDefinitionNames().kind("Widget").plural("widgets")).scope("Namespaced")
                .versions(versions));
        body.setStatus(new V1CustomResourceDefinitionStatus().acceptedNames(new V1CustomResourceDefinitionNames().kind("Widget")));

        String dryRun = "All";
        V1CustomResourceDefinition result = apiInstance.createCustomResourceDefinition(body)
                .dryRun(dryRun)
                .execute();

Results in:

  	SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
Oct 06, 2025 8:25:57 PM okhttp3.internal.platform.Platform log
INFO: --> POST https://10.235.11.65:6443/apis/apiextensions.k8s.io/v1/customresourcedefinitions?dryRun=All
Oct 06, 2025 8:25:57 PM okhttp3.internal.platform.Platform log
INFO: Content-Length: 772
Oct 06, 2025 8:25:57 PM okhttp3.internal.platform.Platform log
INFO: Accept: application/json
Oct 06, 2025 8:25:57 PM okhttp3.internal.platform.Platform log
INFO: Content-Type: application/json
Oct 06, 2025 8:25:57 PM okhttp3.internal.platform.Platform log
INFO: User-Agent: Kubernetes Java Client/24.0.0-SNAPSHOT
Oct 06, 2025 8:25:57 PM okhttp3.internal.platform.Platform log
INFO: 
Oct 06, 2025 8:25:57 PM okhttp3.internal.platform.Platform log
INFO: {"apiVersion":"apiextensions.k8s.io/v1","kind":"CustomResourceDefinition","metadata":{"annotations":{},"finalizers":[],"labels":{},"name":"widgets.example.com","ownerReferences":[]},"spec":{"group":"example.com","names":{"categories":[],"kind":"Widget","plural":"widgets","shortNames":[]},"scope":"Namespaced","versions":[{"additionalPrinterColumns":[],"name":"v1","schema":{"openAPIV3Schema":{"allOf":[],"anyOf":[],"definitions":{},"dependencies":{},"enum":[],"oneOf":[],"patternProperties":{},"properties":{},"required":[],"type":"object","x-kubernetes-list-map-keys":[],"x-kubernetes-validations":[]}},"selectableFields":[],"served":true,"storage":true}]},"status":{"acceptedNames":{"categories":[],"kind":"Widget","shortNames":[]},"conditions":[],"storedVersions":[]}}
Oct 06, 2025 8:25:57 PM okhttp3.internal.platform.Platform log
INFO: --> END POST (772-byte body)
Oct 06, 2025 8:25:58 PM okhttp3.internal.platform.Platform log
INFO: <-- 422 https://10.235.11.65:6443/apis/apiextensions.k8s.io/v1/customresourcedefinitions?dryRun=All (619ms)
Oct 06, 2025 8:25:58 PM okhttp3.internal.platform.Platform log
INFO: audit-id: 0801aeff-97de-422d-a6d6-9c8b2b6890af
Oct 06, 2025 8:25:58 PM okhttp3.internal.platform.Platform log
INFO: cache-control: no-cache, private
Oct 06, 2025 8:25:58 PM okhttp3.internal.platform.Platform log
INFO: content-type: application/json
Oct 06, 2025 8:25:58 PM okhttp3.internal.platform.Platform log
INFO: warning: 299 - "unknown field \"spec.versions[0].selectableFields\""
Oct 06, 2025 8:25:58 PM okhttp3.internal.platform.Platform log
INFO: x-kubernetes-pf-flowschema-uid: 7cd60bae-d919-4fdd-886f-ced99e44cb01
Oct 06, 2025 8:25:58 PM okhttp3.internal.platform.Platform log
INFO: x-kubernetes-pf-prioritylevel-uid: e3c32b39-315a-44b3-a223-702ebb709445
Oct 06, 2025 8:25:58 PM okhttp3.internal.platform.Platform log
INFO: content-length: 538
Oct 06, 2025 8:25:58 PM okhttp3.internal.platform.Platform log
INFO: date: Mon, 06 Oct 2025 14:06:23 GMT
Oct 06, 2025 8:25:58 PM okhttp3.internal.platform.Platform log
INFO: 
Oct 06, 2025 8:25:58 PM okhttp3.internal.platform.Platform log
INFO: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"CustomResourceDefinition.apiextensions.k8s.io \"widgets.example.com\" is invalid: spec.validation.openAPIV3Schema.dependencies: Forbidden: dependencies is not supported","reason":"Invalid","details":{"name":"widgets.example.com","group":"apiextensions.k8s.io","kind":"CustomResourceDefinition","causes":[{"reason":"FieldValueForbidden","message":"Forbidden: dependencies is not supported","field":"spec.validation.openAPIV3Schema.dependencies"}]},"code":422}

Oct 06, 2025 8:25:58 PM okhttp3.internal.platform.Platform log
INFO: <-- END HTTP (538-byte body)
Exception when calling ApiextensionsV1Api#createCustomResourceDefinition
Status code: 422
Reason: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"CustomResourceDefinition.apiextensions.k8s.io \"widgets.example.com\" is invalid: spec.validation.openAPIV3Schema.dependencies: Forbidden: dependencies is not supported","reason":"Invalid","details":{"name":"widgets.example.com","group":"apiextensions.k8s.io","kind":"CustomResourceDefinition","causes":[{"reason":"FieldValueForbidden","message":"Forbidden: dependencies is not supported","field":"spec.validation.openAPIV3Schema.dependencies"}]},"code":422}

Response headers: {audit-id=[0801aeff-97de-422d-a6d6-9c8b2b6890af], cache-control=[no-cache, private], content-length=[538], content-type=[application/json], date=[Mon, 06 Oct 2025 14:06:23 GMT], warning=[299 - "unknown field \"spec.versions[0].selectableFields\""], x-kubernetes-pf-flowschema-uid=[7cd60bae-d919-4fdd-886f-ced99e44cb01], x-kubernetes-pf-prioritylevel-uid=[e3c32b39-315a-44b3-a223-702ebb709445]}
io.kubernetes.client.openapi.ApiException: Message: 
HTTP response code: 422
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"CustomResourceDefinition.apiextensions.k8s.io \"widgets.example.com\" is invalid: spec.validation.openAPIV3Schema.dependencies: Forbidden: dependencies is not supported","reason":"Invalid","details":{"name":"widgets.example.com","group":"apiextensions.k8s.io","kind":"CustomResourceDefinition","causes":[{"reason":"FieldValueForbidden","message":"Forbidden: dependencies is not supported","field":"spec.validation.openAPIV3Schema.dependencies"}]},"code":422}

HTTP response headers: {audit-id=[0801aeff-97de-422d-a6d6-9c8b2b6890af], cache-control=[no-cache, private], content-length=[538], content-type=[application/json], date=[Mon, 06 Oct 2025 14:06:23 GMT], warning=[299 - "unknown field \"spec.versions[0].selectableFields\""], x-kubernetes-pf-flowschema-uid=[7cd60bae-d919-4fdd-886f-ced99e44cb01], x-kubernetes-pf-prioritylevel-uid=[e3c32b39-315a-44b3-a223-702ebb709445]}
	at io.kubernetes.client.openapi.ApiClient.handleResponse(ApiClient.java:1154)
	at io.kubernetes.client.openapi.ApiClient.execute(ApiClient.java:1067)
	at io.kubernetes.client.openapi.apis.ApiextensionsV1Api.createCustomResourceDefinitionWithHttpInfo(ApiextensionsV1Api.java:158)
	at io.kubernetes.client.openapi.apis.ApiextensionsV1Api$APIcreateCustomResourceDefinitionRequest.execute(ApiextensionsV1Api.java:259)
	at com.example.crd.CrdTest.main(CrdTest.java:43)

Expected behavior The expectation is that the above code would create the CRD.

ardhendusaha avatar Oct 06 '25 15:10 ardhendusaha

That error indicates that the syntax for your API resource is wrong. Have you tried submitting it via kubectl does it work if you do that? If it works with kubectl then there is probably a bug in the client somewhere, if it doesn't work with kubectl then the problem is in the resource body.

Also, fwiw, those docs are generated by the code generator, and honestly not very good. You should look at the examples here instead:

https://github.com/kubernetes-client/java/tree/v24.0.0/examples/examples-release-latest/src/main/java/io/kubernetes/client/examples

brendandburns avatar Oct 08 '25 14:10 brendandburns

@brendandburns Thanks for your quick response. I am sharing my findings below.

While using the Kubernetes Java client (ApiextensionsV1Api) version 20.0.0 or greater to generate and apply a CustomResourceDefinition, we encountered the following issues during kubectl apply:

Errors Observed

selectableFields

Error: Error from server (BadRequest): CustomResourceDefinition in version "v1" cannot be handled as a CustomResourceDefinition: strict decoding error: unknown field "spec.versions[0].selectableFields"

Resolution: Removing selectableFields allowed the CRD to proceed further.

dependencies

Error: The CustomResourceDefinition "widgets.example.com" is invalid: spec.validation.openAPIV3Schema.dependencies: Forbidden: dependencies is not supported

Resolution: Removing dependencies allowed the CRD to be created successfully.

Working Scenario

After removing both selectableFields and dependencies, the CRD was successfully created using:

kubectl apply -f java-client-crd.json

Observation These fields appear to be introduced after the 19.0.0 release but are not yet supported or properly handled in client versions 20.0.0 or greater. This results in CRD generation that is incompatible with the Kubernetes API server.

💡 Suggested Fix

  • The Java client should avoid generating unsupported fields by default or provide configuration options to exclude them.

  • Documentation should clarify which CRD fields are supported per client version.

Attaching the generated and working CRD files for reference.

java-client-crd-generated.json java-client-crd-working.json

Refer the below output from the cluster for the above crds.

Image

ardhendusaha avatar Oct 09 '25 13:10 ardhendusaha

Ah, I see the problem, you are targetting Kubernetes 1.28.0 but Java client version 24.0 is generated at a later version of Kubernetes (1.33) those fields were introduced in later versions of the Kubernetes API (e.g. https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/4358-custom-resource-field-selectors/README.md)

You need to either update your Kubernetes cluster, or you need to use an older version of the client.

brendandburns avatar Oct 09 '25 17:10 brendandburns

@brendandburns Thanks!

We have tried with Kubernetes v1.32.3 upstream cluster and noticed the same error. The goal is to communicate with different versions of Kubernetes clusters using the latest java client without CVEs.

JazzJam avatar Oct 09 '25 20:10 JazzJam

@brendandburns It would be very helpful if you could provide a workaround to address this issue. Even a temporary solution or a pointer in the right direction would be much appreciated.

ardhendusaha avatar Oct 13 '25 16:10 ardhendusaha

Each client version is generated for a particular version of Kubernetes, while generally there is comptability with earlier versions, sometimes there are situations like this where there is a breaking change in the API.

This client library maintains the same N-2 version support policy that Kubernetes does, so we support versions 34, 33 and 32, if you need dependencies updated to address CVEs in those versions, feel free to send requests for cherry-picks (or even better cherry-pick PRs) and we can release new versions. For earlier versions, they are out of the support window.

You can probably use the dynamic client: https://github.com/kubernetes-client/java/blob/master/examples/examples-release-latest/src/main/java/io/kubernetes/client/examples/DynamicClientExample.java

To achieve what you want since it allows you to manipulate objects with raw JSON instead of structured objects.

brendandburns avatar Oct 15 '25 15:10 brendandburns