[design-proposal] Add support for testing

[gscho]

The effortless pattern should support testing of the components and be included as a best practice.

Describe the problem

Currently when building out an effortless project testing is an exercise left up to the user. There should be a best practice decided on and included in this repository to improve this experience because it's pretty important to end users.

Software Version

N/a

Possible Solution

Currently when testing effortless packages we do the following:

1. Build the habitat package
2. Use kitchen-vagrant to create a VM(s) with the results directory synced
3. Install the package on the VMs
4. Run the inspec tests

Open to other suggestions!

[echohack]

Thank you @gscho for opening this issue!

it's our intent to build automated testing that works in concert with our new automated ci processes.

It's unlikely that this will use test-kitchen however. We are using expeditor and build kite, which are standard Chef-OSS repo CI to run our automated PR checks, and these mechanisms are best suited for running automated tests as well. In fact, the CI is already hooked up to do exactly that.

habitat-sh/core-plans has already done much of the hard work for us in defining test standards, and so we may apply those here as well, albeit with a couple of modifications and simplification (since we don't have to managed 1000 packages).

We will need to test a few things:

1. The public API that we provide to consumers of the scaffolding packages.
2. The build process of the scaffolding.
3. The build process of the consuming package.
4. The runtime and configuration data of chef-infra-client and chef-inspec.

This will need to be in the format of an official design proposal to be accepted and discussed. I'll see if I can get around to doing that soon. 🗡

[gscho]

Is the purpose of this repo that it serves as a sample to users on how to properly implement the pattern?

I'd hate for the testing portion to be Chef organization specific meaning it can't be easily translated to others not using build-kite and expeditor.

Also as far as I know the core-plans testing do not include any sort of smoke/integration testing which I think is vital, just unit testing via BATS. Would you see integration tests as included in #4?

[echohack]

@gscho There are examples available in docs/examples.

The test portion of this is coming soon! It should be portable between your CI solution of choice. I'll make a formal proposal -- ideally we will be using bats for testing bash, pester for testing powershell, and using InSpec to test any integration or runtime.

Integration tests for Effortless are critical because there is a significant runtime component to the scaffoldings.