'$ref' created in OpenAPI CRD schema

[xSAVIKx]

Hi, I'm trying to build a k8s operator using kubebuilder and face an issue similar to one reported in https://github.com/kubernetes-sigs/controller-tools/issues/324.

A CRD definition is generated with $ref being used instead of flattening the schema.

I'm using latest v0.18.0 controller-gen.

Here's the spec structure:

import (
	istioapiv1alpha3 "istio.io/api/networking/v1alpha3"
)

// Target defines the source resource to merge with
type Target struct {
	// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
	// Important: Run "make" to regenerate code after modifying this file

	// +kubebuilder:validation:Required
	Name string `json:"name,omitempty"`

	Namespace string `json:"namespace,omitempty"`
}

// IstioVirtualServiceMergeSpec defines the desired state of IstioVirtualServiceMerge.
type IstioVirtualServiceMergeSpec struct {
	// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
	// Important: Run "make" to regenerate code after modifying this file

	// +kubebuilder:validation:Required
	Target Target `json:"target"`

	// +kubebuilder:validation:Required
	Patch istioapiv1alpha3.VirtualService `json:"patch"`
}

And here's the CRD being generated:

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.18.0
  name: istiovirtualservicemerges.istiovsmerger.xsavikx.github.io
spec:
  group: istiovsmerger.xsavikx.github.io
  names:
    kind: IstioVirtualServiceMerge
    listKind: IstioVirtualServiceMergeList
    plural: istiovirtualservicemerges
    singular: istiovirtualservicemerge
  scope: Namespaced
  versions:
  - name: v1beta1
    schema:
      openAPIV3Schema:
        description: IstioVirtualServiceMerge is the Schema for the istiovirtualservicemerges
          API.
        properties:
          apiVersion:
            description: |-
              APIVersion defines the versioned schema of this representation of an object.
              Servers should convert recognized schemas to the latest internal value, and
              may reject unrecognized values.
              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
            type: string
          kind:
            description: |-
              Kind is a string value representing the REST resource this object represents.
              Servers may infer this from the endpoint the client submits requests to.
              Cannot be updated.
              In CamelCase.
              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
            type: string
          metadata:
            type: object
          spec:
            description: IstioVirtualServiceMergeSpec defines the desired state of
              IstioVirtualServiceMerge.
            properties:
              patch:
                $ref: '#/definitions/istio.io~1api~1networking~1v1alpha3~0VirtualService'
              target:
                description: Target defines the source resource to merge with
                properties:
                  name:
                    type: string
                  namespace:
                    type: string
                required:
                - name
                type: object
            required:
            - patch
            - target
            type: object
          status:
            description: IstioVirtualServiceMergeStatus defines the observed state
              of IstioVirtualServiceMerge.
            properties:
              HandledRevision:
                type: string
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}

controller gen CLI call:

controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases

and here's the output I receive:

/home/xsavikx/go/pkg/mod/istio.io/[email protected]/networking/v1alpha3/workload_entry.pb.go:196:1: expected comma, got "\").startsWith('unix://')) ? !has(self.ports) : true\"" (at <input>:1:67)
/home/xsavikx/go/pkg/mod/istio.io/[email protected]/networking/v1alpha3/workload_entry.pb.go:196:1: expected comma, got "\").startsWith('unix://')) ? !has(self.ports) : true\"" (at <input>:1:67)
istio.io/api/networking/v1alpha3:-: unknown type "istio.io/api/networking/v1alpha3".VirtualService
istio.io/api/networking/v1alpha3:-: unknown type "istio.io/api/networking/v1alpha3".VirtualService
github.io/xSAVIKx/istio-vs-merger/api/v1beta1:-: unable to locate schema for type "istio.io/api/networking/v1alpha3".VirtualService
Error: not all generators ran successfully
run `controller-gen rbac:roleName=manager-role crd webhook paths=./... output:crd:artifacts:config=config/crd/bases -w` to see all available markers, or `controller-gen rbac:roleName=manager-role crd webhook paths=./... output:crd:artifacts:config=config/crd/bases -h` for usage
make: *** [Makefile:46: manifests] Error 1

[xSAVIKx]

cc @sbueringer, @alvaroaleman.

I'd appreciate some suggestions on this. Happy to try and fix it myself, just not very experienced with go yet.

[sbueringer]

To be honest, I also have no idea why that happens. Maybe the Istio folks have some experience with using controller-gen with their APIs?

[sbueringer]

Took a closer look at

/home/xsavikx/go/pkg/mod/istio.io/[email protected]/networking/v1alpha3/workload_entry.pb.go:196:1: expected comma, got "\").startsWith('unix://')) ? !has(self.ports) : true\"" (at <input>:1:67)
/home/xsavikx/go/pkg/mod/istio.io/[email protected]/networking/v1alpha3/workload_entry.pb.go:196:1: expected comma, got "\").startsWith('unix://')) ? !has(self.ports) : true\"" (at <input>:1:67)

This marker seems to be simply invalid? https://github.com/istio/api/blob/master/networking/v1alpha3/workload_entry.pb.go#L197

// +kubebuilder:validation:XValidation:message="UDS may not include ports",rule="(default(self.address, "").startsWith('unix://')) ? !has(self.ports) : true"

I think the "" within rule doesn't work. I'm not sure how it works for Istio. I would expect that some sort of escaping is necessary

[xSAVIKx]

Thanks for taking a look @sbueringer.

It looks like Istio was never going to support builds with kubebuilder: https://github.com/istio/api/issues/3363#issuecomment-2479327140

Do you know if there's a way to let controller-gen skip a particular field during the generation or maybe use custom-provided schema instead of generating one? Then I'd be able to e.g. copy Istio-generated schema and include it manually

[sbueringer]

There should be a schemaless marker. Maybe that one helps

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten