What would be the specs for Integration Test environments?

[lucdekens]

There are many builds of vSphere (ESXi and VCSA) out there that are still supported. What would be the approach for running Integration Tests?

• against the latest builds

• against the oldest builds

• against all supported builds

• against all possible combinations of builds (including mixed environments)

We don't, at least most of us don't, have the test environments available that I would assume the PowerCLI Dev Team has.

Would creating and verifying correctness of said Integration Tests once, be sufficient for submitters of PRs? And would the PowerCLI Dev Team run the same Integration Tests than against their battery of available/possible environments for final validation?

That sounds costly, more so since we are dealing with Open Source code.

[lucdekens]

Related to this, how and where do we keep track of build versions of the instance represented in a class? Would that go in the base class?

And how would we test for a build prereq in a resource? For example, a feature that was introduced in build x.y.z.

[dmilov]

Good question. The goal of the integration tests is to prove the new or updated resource works against real server connection. Since PowerCLI stays between the DSC Resource and the Server and PowerCLI covers the Client->Server matrix you're talking about, it's not needed you to run integration tests that cover this matrix.

Testing against whatever server version you have and rely on PowerCLI to guarantee it would work against all other supported version. There is a risk of exposing feature that is available from certain version and above that will produce errors for lower versions. If you're aware of this for the resource or property you're working on please put this in the description comments.

We still don't have compatibility matrix for the DSC Resources but when we decide to publish release version in PS Gallery we would use the integration tests to cover the defined matrix.

[dmilov]

o this, how and where do we keep track of build versions of the instance represented in a class? Would that go in the base class?

And how would we test for a build prereq in a resource? For example, a feature that was introduced in build x.y.z.

We are working on Travis build on pull request which will run the unit tests and in addition it will generate new build number of the module. There technology time to enable Travis on this repo. hopefully we'lll have it soon.

[lucdekens]

The build number of the module is indeed another feature that would preferable be automated.

But I was more thinking off the ESXi and VCSA versions. We would need to store these somewhere in the class properties as well. And perhaps it would be best to have these in the base class, or as low in the inheritance hierarchy as possible. Otherwise other contributors might come up with their own solutions, and we risk having multiple properties with similar content.

[dmilov]

But I was more thinking off the ESXi and VCSA versions. We would need to store these somewhere in the class properties as well. And perhaps it would be best to have these in the base class, or as low in the inheritance hierarchy as possible. Otherwise other contributors might come up with their own solutions, and we risk having multiple properties with similar content.

Got it. Something we have to figure out 👍