Add end-to-end testing

[melody-universe]

I've started on an end-to-end testing suite prior to release of v9 and wanted to open an issue here to track it and invite anyone into the discussion.

The design goal is to test the end-to-end functionality of this project without modifying any of its existing code. So in order to run these tests, a contributor would have to create a separate repository and generate a PAT for the tests to run in that repository.

Also, to test pull requests from external repositories (logic that needs to be handled differently in the code of this project), a contributor will have to create a machine user account and provide a PAT for it in the forked repository.

I'm going to need to document a lot of this and am of course open to any contributions or feedback (even a dev session if anyone's interested!). You can look at my fork here and observe the results of the tests here.

EDIT: This is a pretty ambitious undertaking; I'm going to shift my efforts towards delivering what I have now and then invite the community to contribute.

To Do

• [x] Parallelize tests.
• ~~Add test for pull requests from forks.~~
• [ ] Add documentation for setting up a fork.
• ~~Test set-output for newTag.~~
• ~~Test custom user/email configuration.~~
• ~~Add test for multiple commits in a single workflow execution.~~
• [ ] Refactor the testing codebase.

[rosshamish]

Hey @melody-universe , are you interested in contributions in this area? I leveraged this test suite to test my changes in #143 , I could contribute back the changes I made to get the test running in my fork.

Some of my changes:

• added a Testing section to README.md describing how the test suite works, and how to add a new test, with an example
• couple fixes to some expected blocks in the test config.yml to match my expectations (and get them passing)

I'm interested in getting these tests running locally, rather than needing a blank public Github repo. This would make them run way faster in theory, and would also drop the heavy requirement of creating and keeping a test repo. I think this might be doable with a combination of

1. https://github.com/nektos/act which allows for running github actions locally, and
2. using a local "bare repo" as the remote rather than a github.com remote

[melody-universe]

@rosshamish Sorry for the very delayed response. I've been out of the Node world for a while. I agree, spinning this up in a GitHub environment has felt pretty hacky. I think I tried to do this in act but wasn't able to get it working. If you still have anything you'd like to submit to help push this one forward, please feel free!