azure-service-operator icon indicating copy to clipboard operation
azure-service-operator copied to clipboard

Reduce the noise in our golden tests

Open theunrepentantgeek opened this issue 3 years ago • 6 comments
trafficstars

Describe the current behavior

Many of our golden tests (the ones driven by test data under v2\tools\generator\internal\codegen\testdata) are checking for the structure of the final data types, but also contain a lot of other generated code that's simply noise.

Describe the improvement

Reduce the amount of generated code so that the data structures become the focus.

Additional context

We can't just filter out all functions - tried that, it suppressed too much.

We also can't just pull out some of the pipeline stages. For one, they're often prerequisites of other stages that we do want to test. For another, a lot of our stages are ONLY tested by these golden tests (they don't have isolated unit tests of their own). The more our test pipeline resembles the actual one we use for code generation the better, because we rule out unexpected interactions between the stages.

theunrepentantgeek avatar Mar 10 '22 01:03 theunrepentantgeek

Our test package already includes a fluent style API for doing assertions:

test.AssertPackagesGenerateExpectedCode(
	t, 
	finalState.definitions, 
	test.DiffWithTypes(initialState.Definitions()))

We might add more options to allow removing specific functions and/or interface implementations before generating the golden files:

test.AssertPackagesGenerateExpectedCode(
	t, 
	finalState.definitions,
	test.RemoveAll(&PropertyAssignmentFunction{}),
	test.RemoveAll(&PivotConversionFunction{}),
	test.RemoveAll(&OriginalGVKFunction{}))

These would work by pruning any functions (or interface implementations) that match the specified types.

theunrepentantgeek avatar Mar 14 '22 21:03 theunrepentantgeek

This is something we're still interested in doing, but its not a high priority right now.

matthchr avatar Jul 18 '22 23:07 matthchr

Still interested in this, but it's not a high priority currently

matthchr avatar Jan 09 '23 23:01 matthchr

This is still something we're interested in, and related to getting rid of gojsonschema too (#1592)

matthchr avatar Apr 08 '24 23:04 matthchr