gruntwork-io
WIP: "tspec" - Add a BDD-style interface to Terratest
This PR adds a BDD-style interface to Terratest which means you can write infrastructure tests in plain-English.
RFC
Terratest is great and really powerful, but since it's written in Go, it can sometimes be a significant learning curve for a lot of people to properly test their infrastructure. Imagine if you could write infrastructure tests in plain-English, but take advantage of all of the Terratest modules under the hood? This PR adds a BDD-style interface to Terratest which means you can write infrastructure tests without writing a single line of code.
Consider the Hello, World test for Terratest. It's 25+ lines for the simplest use case. Using tspec you could write a feature test as follows:
Feature: Terraform Hello World
Scenario: Run a simple test
Given the Terraform module at "./examples/terraform-hello-world-example"
When I run "terraform apply"
Then the "hello_world" output is "Hello, World!"
Or to test something slightly more complicated you could write a feature test along the lines of:
Feature: Terraform AWS Example
Scenario: Run a simple test on AWS
Given the Terraform module at "./examples/terraform-aws-example"
And an input variable named "instance_name" with the value "test-instance-1"
And an environment variable named "AWS_DEFAULT_REGION" with the value "us-east-1"
When I run "terraform apply"
Then the "instance_id" output should match "i-[0-9a-z]+(\,i-[0-9a-z]+)*"
When you execute tspec it will automatically copy the specified Terraform module to a temporary path, pass in the target parameters and ensure the tests pass as expected using Terratest methods under the hood.
PoC
Under the hood, tspec borrows heavily from the Godog Cucumber library but includes a library of common step definitions so users don't need to write any Go code.
Note: Unfortunately this is a rather large PR, so this RFC should serve as a potential discussion as to how we can "chop" this tool down into the smallest possible product increment.
TODO
- [ ] Support for Background grammar.
- [ ] Add more grammar for tools like Packer & Docker.
Open Questions
- Should we remove all of the various formatters like
junit,eventetc from the first version? - What should we name this?
tspec,infraspec?
@brikis98 hey Jim, thanks for a first pass! I feel you may have missed my remark in the RFC: the code borrows heavily from the Godog Cucumber library. I think a good approach now is possibly how I've extended this library CLI/tool and maybe we should focus on reviewing my custom parts. An additional step would be figuring out how to reduce the amount of code here. This would be super easy for me to explain on a call, so I'm wondering if we should briefly chat on Friday during Ticket Jam.
p.s I'm not going to respond to your comments in the meantime.
@brikis98 hey Jim, thanks for a first pass! I feel you may have missed my remark in the RFC: the code borrows heavily from the Godog Cucumber library.
I saw that, but coudn't figure out which parts were borrowed, and which are unique? Also, is it possible to use those parts directly from godog, pulling them in as dependencies?
This would be super easy for me to explain on a call, so I'm wondering if we should briefly chat on Friday during Ticket Jam.
Sure, chatting live is fine too.
p.s I'm not going to respond to your comments in the meantime.
What a jerk 😄
@brikis98 actually on 2nd thoughts I'll leave a few comments here ahead of the call:
I saw that, but couldn't figure out which parts were borrowed, and which are unique? Also, is it possible to use those parts directly from Godog, pulling them in as dependencies?
Yes definitely! I was unsure how much of the library I'd actually need to modify, but now I have a better sense I'm wondering if we should simply import the parts we need and only sprinkle in our custom logic? The downsides would be:
- We need to add additional imports to Terratest.
- You've mentioned some parts of the code aren't very clear to you, e.g: their interfaces and two-letter variable names. If we import the code, we'll miss the opportunity to throw away the parts we don't need and also explain things more clearly.
On a more positive note, Godog is MIT-licensed, so we can either import the parts we need or vendor them directly into Terratest.
I saw that, but couldn't figure out which parts were borrowed, and which are unique? Also, is it possible to use those parts directly from Godog, pulling them in as dependencies?
Yes definitely! I was unsure how much of the library I'd actually need to modify, but now I have a better sense I'm wondering if we should simply import the parts we need and only sprinkle in our custom logic? The downsides would be:
- We need to add additional imports to Terratest.
- You've mentioned some parts of the code aren't very clear to you, e.g: their interfaces and two-letter variable names. If we import the code, we'll miss the opportunity to throw away the parts we don't need and also explain things more clearly.
On a more positive note, Godog is MIT-licensed, so we can either import the parts we need or vendor them directly into Terratest.
Adding imports is definitely preferred. In fact, this is a standalone CLI tool, so those imports shouldn't affect anyone not directly using the CLI tool.
Hello :wave:
Did the idea of using Terratest in a BDD framework get off the ground beyond what's in this issue?
@francis-io Hi Mark, sorry for the delayed reply. I've been away on vacation and am slowly getting caught up on everything. Short answer - No. Long answer: We've decided to prioritise other products and projects so this idea was temporarily put on hold. Also I don't have an ETA for when we'd pick it up again. You can somewhat try and use or take advantage of what I've pushed in this branch, but we wouldn't be able to offer any official support for it at this stage.