Cloud-CV
Integrate Postman API Tests into GitHub Actions CI Pipeline
Description:
To improve backend API reliability, we should integrate Postman API tests into EvalAI’s GitHub Actions CI pipeline. This will ensure that core API endpoints remain functional and are not unintentionally broken by future pull requests — especially since current CI primarily validates backend unit tests and frontend builds.
This was originally proposed for Travis CI, but since we are migrating to GitHub Actions, the integration should be done there.
Plan:
✅ Step 1: Create Postman Test Assets
-
[ ] Create a folder at the root of the repo (e.g.,
postman-tests/) -
[ ] Add the following files:
-
evalai-api-tests.postman_collection.json: Contains the API test cases. -
evalai-env.postman_environment.json: Holds base URL, auth tokens, etc.
-
✅ Step 2: Add GitHub Actions Workflow
-
[ ] Use a GitHub Actions workflow to:
-
Set up Node.js
-
Install [Newman](https://github.com/postmanlabs/newman), the CLI tool for Postman
-
Run the collection:
newman run postman-tests/evalai-api-tests.postman_collection.json \ -e postman-tests/evalai-env.postman_environment.json -
Fail the build if any API test fails
-
Additional Notes:
- This should be coordinated with or follow the [Travis → GitHub Actions migration](https://github.com/Cloud-CV/EvalAI/pull/4576).
Hi! I'm a beginner contributor interested in DevOps and CI, and this issue looks like a great learning opportunity. May I please take this up and try it out? I may ask for help along the way. 🙂
Hi @RishabhJain2018 , I’d love to take this up if it’s still available. I can set up the Postman tests and integrate them with GitHub Actions as described. Could you please assign this issue to me?
