Acceptance test run in PRs

[ceilfors]

Story

As a contributor, I would like to run acceptance test before my Pull Request is merged, So that I can gain more confidence that my commits are not breaking anything

Acceptance criteria

Acceptance test is running in pull request

Additional context

• This issue is related to #99
• The concern for contributors at the moment is the need for an AWS account to run this
• Is there a better way for us to gain confidence without running the test in the cloud?

[hugosenari]

How we can do this without security problems?

Ie.:

1. Sensitive data leak (accountId, secrets, etc);
2. DoS 'attacks' (high usage of AWS services could lead to higher bills)

[ceilfors]

Continuing the conversation we had at #435 here @hugosenari. The high bill is what I'm worried about. If we run the acceptance test on every PR, I will have to tank the bill when it exceeds the free limit, and this project is running without any funding!

Edit: OK, I just found: https://aws.amazon.com/blogs/opensource/aws-promotional-credits-open-source-projects/. I'm still worried about DoS attack though, attackers can create many branches and file many PRs. I wonder if there's a 'non-automatic' way of running the test in PR i.e. maintainers can click on the button on PR to run the test. Alternatively, we can just run acceptance test only in master for now.

[hugosenari]

Run only in master for now :wink:

After that, CircleCI can hold a workflow for a manual approval.

[tschoffelen]

Closing this for now - I've updated the acceptance test to run as a requirement before the merge can succeed (using the merge_group trigger in the GitHub Actions Workflow).

Hopefully that gives us the best of both worlds, still keeping the executions down, but running it before things can be merged.