convoy
convoy copied to clipboard
add Mayhem for API testing as a github workflow
Add a github action to run Mayhem for API (which is a free automated testing tool for http APIs) against the api: given the swagger spec and api server, it generates and runs random test payloads, looking for exceptional behavior.
Disclosure: working on this tool is my day job!
As with any github workflow, test results can be consumed through github. Alternatively, Mayhem for API also has a dashboard, over here: https://mayhem4api.forallsecure.com/mayhemheroes/convoy.
Currently this finds a bunch of Internal Server Errors and Server Crashes that can be pretty easily triggered, as well as lots of mismatches between the openapi spec and the empirical responses (these warnings can be disabled if they're not interesting, but might be useful to keep your spec and code aligned, depending on how you maintain the specs.)
To make it work in the upstream repo, there's a bit of out-of-PR effort to integrate:
- sign up for a free mapi account, creating a frain-dev organization
- create a mapi service account within the frain-dev organization
- modify the mapi.yaml action in this PR to point at your mapi organization
- add the service account's API token to github as a repository secret named MAPI_TOKEN
Deploy Preview for getconvoy canceled.
Name | Link |
---|---|
Latest commit | 3b46491405e29324b77640753e9f4e9370cdac1d |
Latest deploy log | https://app.netlify.com/sites/getconvoy/deploys/62a27cea9c344500072dd847 |
Hi @dlowe,
Thanks for raising this PR. This looks interesting. But I don't think we want to merge and maintain this test suite right now. Hopefully, you can submit this PR again sometime soon, early next year, and we can re-assess!
Thanks!