upjet icon indicating copy to clipboard operation
upjet copied to clipboard

Flaky TestNewKubernetesSource/Successful unit test

Open lsviben opened this issue 2 years ago • 1 comments

What happened?

Seems that this test is flaky, seen it fail mutiple times over different PRs

=== RUN   TestNewKubernetesSource/Successful
    kubernetes_test.go:122: 
        Next(...): -want, +got:
          []migration.UnstructuredWithMetadata{
        - 	{
        - 		Object: unstructured.Unstructured{
        - 			Object: map[string]any{
        - 				"apiVersion": string("ec2.aws.crossplane.io/v1beta1"),
        - 				"kind":       string("VPC"),
        - 				"metadata":   map[string]any{...},
        - 				"spec":       map[string]any{...},
        - 			},
        - 		},
        - 		Metadata: migration.Metadata{Category: s"managed"},
        - 	},
          	{Object: {Object: {"apiVersion": string("azure.crossplane.io/v1beta1"), "kind": string("ResourceGroup"), "metadata": map[string]any{"name": string("example-resources")}, "spec": map[string]any{"forProvider": map[string]any{"location": string("West Europe")}}}}, Metadata: {Category: s"managed"}},
        + 	{
        + 		Object: unstructured.Unstructured{
        + 			Object: map[string]any{
        + 				"apiVersion": string("ec2.aws.crossplane.io/v1beta1"),
        + 				"kind":       string("VPC"),
        + 				"metadata":   map[string]any{...},
        + 				"spec":       map[string]any{...},
        + 			},
        + 		},
        + 		Metadata: migration.Metadata{Category: s"managed"},
        + 	},
          }
--- FAIL: TestNewKubernetesSource (0.00s)
    --- FAIL: TestNewKubernetesSource/Successful (0.00s)

How can we reproduce it?

I assume just running unit tests multiple times should trigger it. I see if fairly often on PRs

lsviben avatar Sep 07 '23 13:09 lsviben

I can confirm this flaky behavior by simply triggering the unit tests multiple times: repeat 10 make test | grep TestNewKubernetesSource

CleanShot 2023-10-10 at 14 39 35@2x

CleanShot 2023-10-10 at 14 41 47@2x

jeanduplessis avatar Oct 10 '23 12:10 jeanduplessis