dotnet-operator-sdk icon indicating copy to clipboard operation
dotnet-operator-sdk copied to clipboard
verified publisher icon buehler

Complex Entities cause stack overflow when generating CRDs

Open bwarthur opened this issue 3 years ago • 3 comments

Is your feature request related to a problem? Please describe. I have several different classes that were auto generated from these protobufs that appear to contain circular references. When I set one of these classes as the Spec property for an entity I will get a stack overflow when doing a build. I am currently working around this by setting IgnoreEntity attribute on the kubernetes entity and manually creating/updating the CRD.

Because of how complex the spec type is, I have just been setting the type as object and adding x-kubernetes-preserve-unknown-fields: true to the property in the CRD schema.

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: clusters.xds
spec:
  group: xds
  names:
    kind: Cluster
    listKind: ClusterList
    plural: clusters
    singular: cluster
  scope: Namespaced
  versions:
  - name: v1alpha1
    schema:
      openAPIV3Schema:
        properties:
          status:
            properties: {}
            type: object
          spec:
            properties: {}
            type: object
            x-kubernetes-preserve-unknown-fields: true
        type: object
    served: true
    storage: true
    subresources:
      status: {}

[KubernetesEntity(
    ApiVersion = "v1alpha1",
    Kind = "Cluster",
    Group = "xds",
    PluralName = "clusters")]
public class V1Alpha1Cluster : CustomKubernetesEntity, IStatus<V1Alpha1ClusterStatus>
{
    public V1Alpha1Cluster()
    {
    }

    [PreserveUnknownFields]
    public Envoy.Config.Cluster.V3.Cluster? Spec { get; set; }

    public V1Alpha1ClusterStatus Status { get; set; }
}

public class V1Alpha1ClusterStatus
{

}

Describe the solution you'd like Ideally I would like an attribute I can set on my Spec properties that will create an empty properties and set type: object and x-kubernetes-preserve-unknown-fields: true. A solution could be to add an IgnoreEntityPropertyAttribute and if that is set in combination with PreserveUnknownFieldsAttribute then the generated CRD for that property would look like:

spec:
  properties: {}
  type: object
  x-kubernetes-preserve-unknown-fields: true

Additional context For reference, below is a shortened version of the stack trace for the stack overflow exception:

1>Target GenerateAfterBuild:
1>  Target GenerateCrds:
1>    Generating CRDs
1>    Configuration path: ..\VW.Xds.Kubernetes.Operator\\config\crds
1>    Stack overflow.
1>       at System.ModuleHandle.ResolveType(System.Runtime.CompilerServices.QCallModule, Int32, IntPtr*, Int32, IntPtr*, Int32, System.Runtime.CompilerServices.ObjectHandleOnStack)
1>       at System.ModuleHandle.ResolveTypeHandle(Int32, System.RuntimeTypeHandle[], System.RuntimeTypeHandle[])
1>       at System.Reflection.RuntimeModule.ResolveType(Int32, System.Type[], System.Type[])
1>       at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(System.Reflection.MetadataToken, System.Reflection.MetadataImport ByRef, System.Reflection.RuntimeModule, System.Reflection.MetadataToken, System.RuntimeType, Boolean, ListBuilder`1<System.Object> ByRef, System.RuntimeType ByRef, System.IRuntimeMethodInfo ByRef, Boolean ByRef)
1>       at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1<System.Object> ByRef, System.Reflection.RuntimeModule, Int32, System.RuntimeType, Boolean, ListBuilder`1<System.Object>)
1>       at System.Reflection.CustomAttribute.GetCustomAttributes(System.RuntimeType, System.RuntimeType, Boolean)
1>       at System.RuntimeType.GetCustomAttributes(System.Type, Boolean)
1>       at System.Attribute.GetCustomAttributes(System.Reflection.MemberInfo, System.Type, Boolean)
1>       at System.Reflection.CustomAttributeExtensions.GetCustomAttributes[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Reflection.MemberInfo, Boolean)
1>       at KubeOps.Operator.Entities.Extensions.EntityToCrdExtensions.MapType(System.Type, System.Collections.Generic.IList`1<k8s.Models.V1CustomResourceColumnDefinition>, System.String)
1>       at KubeOps.Operator.Entities.Extensions.EntityToCrdExtensions.MapType(System.Type, System.Collections.Generic.IList`1<k8s.Models.V1CustomResourceColumnDefinition>, System.String)
1>       at KubeOps.Operator.Entities.Extensions.EntityToCrdExtensions.MapProperty(System.Reflection.PropertyInfo, System.Collections.Generic.IList`1<k8s.Models.V1CustomResourceColumnDefinition>, System.String)
1>       at KubeOps.Operator.Entities.Extensions.EntityToCrdExtensions+<>c__DisplayClass18_0.<ProcessType>b__1(System.Reflection.PropertyInfo)
1>       at System.Linq.Enumerable+WhereSelectArrayIterator`2[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Collections.Generic.KeyValuePair`2[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
1>       at System.Collections.Generic.Dictionary`2[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].AddRange(System.Collections.Generic.IEnumerable`1<System.Collections.Generic.KeyValuePair`2<System.__Canon,System.__Canon>>)
1>       at System.Collections.Generic.Dictionary`2[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]..ctor(System.Collections.Generic.IEnumerable`1<System.Collections.Generic.KeyValuePair`2<System.__Canon,System.__Canon>>, System.Collections.Generic.IEqualityComparer`1<System.__Canon>)
1>       at System.Collections.Generic.Dictionary`2[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]..ctor(System.Collections.Generic.IEnumerable`1<System.Collections.Generic.KeyValuePair`2<System.__Canon,System.__Canon>>)
1>       at KubeOps.Operator.Entities.Extensions.EntityToCrdExtensions.ProcessType(System.Type, k8s.Models.V1JSONSchemaProps, System.Collections.Generic.IList`1<k8s.Models.V1CustomResourceColumnDefinition>, System.String)
1>       at KubeOps.Operator.Entities.Extensions.EntityToCrdExtensions.MapType(System.Type, System.Collections.Generic.IList`1<k8s.Models.V1CustomResourceColumnDefinition>, System.String)
1>       at KubeOps.Operator.Entities.Extensions.EntityToCrdExtensions.MapProperty(System.Reflection.PropertyInfo, System.Collections.Generic.IList`1<k8s.Models.V1CustomResourceColumnDefinition>, System.String)
1>       at KubeOps.Operator.Entities.Extensions.EntityToCrdExtensions+<>c__DisplayClass18_0.<ProcessType>b__1(System.Reflection.PropertyInfo)

...

1>       at System.Linq.Enumerable+WhereSelectEnumerableIterator`2[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.ValueTuple`2[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Boolean, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
1>       at System.Linq.Lookup`2[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.ValueTuple`2[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Boolean, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Create(System.Collections.Generic.IEnumerable`1<System.ValueTuple`2<System.__Canon,Boolean>>, System.Func`2<System.ValueTuple`2<System.__Canon,Boolean>,System.__Canon>, System.Collections.Generic.IEqualityComparer`1<System.__Canon>)
1>       at System.Linq.GroupedEnumerable`2[[System.ValueTuple`2[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Boolean, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].GetEnumerator()
1>       at System.Linq.Enumerable+SelectEnumerableIterator`2[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ToList()
1>       at System.Linq.Enumerable.ToList[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Generic.IEnumerable`1<System.__Canon>)
1>       at KubeOps.Operator.Commands.Generators.CrdGenerator+<OnExecuteAsync>d__3.MoveNext()
1>       at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[KubeOps.Operator.Commands.Generators.CrdGenerator+<OnExecuteAsync>d__3, KubeOps, Version=6.1.2.0, Culture=neutral, PublicKeyToken=null]](<OnExecuteAsync>d__3 ByRef)
1>       at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[KubeOps.Operator.Commands.Generators.CrdGenerator+<OnExecuteAsync>d__3, KubeOps, Version=6.1.2.0, Culture=neutral, PublicKeyToken=null]](<OnExecuteAsync>d__3 ByRef)
1>       at KubeOps.Operator.Commands.Generators.CrdGenerator.OnExecuteAsync(McMaster.Extensions.CommandLineUtils.CommandLineApplication)
1>       at System.RuntimeMethodHandle.InvokeMethod(System.Object, System.Span`1<System.Object> ByRef, System.Signature, Boolean, Boolean)
1>       at System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)
1>       at System.Reflection.MethodBase.Invoke(System.Object, System.Object[])
1>       at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention+<InvokeAsync>d__2.MoveNext()
1>       at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention+<InvokeAsync>d__2, McMaster.Extensions.CommandLineUtils, Version=4.0.0.0, Culture=neutral, PublicKeyToken=6f71cb76b82f055d]](<InvokeAsync>d__2 ByRef)
1>       at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention+<InvokeAsync>d__2, McMaster.Extensions.CommandLineUtils, Version=4.0.0.0, Culture=neutral, PublicKeyToken=6f71cb76b82f055d]](<InvokeAsync>d__2 ByRef)
1>       at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.InvokeAsync(System.Reflection.MethodInfo, System.Object, System.Object[])
1>       at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention+<OnExecute>d__1.MoveNext()
1>       at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention+<OnExecute>d__1, McMaster.Extensions.CommandLineUtils, Version=4.0.0.0, Culture=neutral, PublicKeyToken=6f71cb76b82f055d]](<OnExecute>d__1 ByRef)
1>       at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention+<OnExecute>d__1, McMaster.Extensions.CommandLineUtils, Version=4.0.0.0, Culture=neutral, PublicKeyToken=6f71cb76b82f055d]](<OnExecute>d__1 ByRef)
1>       at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.OnExecute(McMaster.Extensions.CommandLineUtils.Conventions.ConventionContext, System.Threading.CancellationToken)
1>       at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention+<>c__DisplayClass0_0+<<Apply>b__0>d.MoveNext()
1>       at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention+<>c__DisplayClass0_0+<<Apply>b__0>d, McMaster.Extensions.CommandLineUtils, Version=4.0.0.0, Culture=neutral, PublicKeyToken=6f71cb76b82f055d]](<<Apply>b__0>d ByRef)
1>       at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention+<>c__DisplayClass0_0+<<Apply>b__0>d, McMaster.Extensions.CommandLineUtils, Version=4.0.0.0, Culture=neutral, PublicKeyToken=6f71cb76b82f055d]](<<Apply>b__0>d ByRef)
1>       at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention+<>c__DisplayClass0_0.<Apply>b__0(System.Threading.CancellationToken)
1>       at McMaster.Extensions.CommandLineUtils.CommandLineApplication+<ExecuteAsync>d__154.MoveNext()
1>       at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[McMaster.Extensions.CommandLineUtils.CommandLineApplication+<ExecuteAsync>d__154, McMaster.Extensions.CommandLineUtils, Version=4.0.0.0, Culture=neutral, PublicKeyToken=6f71cb76b82f055d]](<ExecuteAsync>d__154 ByRef)
1>       at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[McMaster.Extensions.CommandLineUtils.CommandLineApplication+<ExecuteAsync>d__154, McMaster.Extensions.CommandLineUtils, Version=4.0.0.0, Culture=neutral, PublicKeyToken=6f71cb76b82f055d]](<ExecuteAsync>d__154 ByRef)
1>       at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync(System.String[], System.Threading.CancellationToken)
1>       at KubeOps.Operator.HostExtensions.RunOperatorAsync(Microsoft.Extensions.Hosting.IHost, System.String[])
1>       at Program.<Main>$(System.String[])

bwarthur avatar Jan 06 '22 17:01 bwarthur

If I recall it correctly, the logic to generate the CRDs is recursive. Maybe there is some error hidden in there :-) I'll have a look, thank you for the report!

buehler avatar Jan 14 '22 08:01 buehler

FYI this is easy to reproduce by just setting the spec type to JsonElement.

[KubernetesEntity(
    ApiVersion = "v1alpha1",
    Kind = "Cluster",
    Group = "xds",
    PluralName = "clusters")]
public class V1Alpha1Cluster : CustomKubernetesEntity, IStatus<V1Alpha1ClusterStatus>
{
    public V1Alpha1Cluster()
    {
    }

    [PreserveUnknownFields]
    public JsonElement? Spec { get; set; }

    public V1Alpha1ClusterStatus Status { get; set; }
}

public class V1Alpha1ClusterStatus
{

}

Since posting the bug I have also found a better workaround for the issue. I just set the spec type to object and the code generation works as desired giving me an empty object in the CRD yaml.

spec:
  properties: {}
  type: object
  x-kubernetes-preserve-unknown-fields: true

bwarthur avatar Jan 14 '22 15:01 bwarthur