fides
fides copied to clipboard
parallelize api and non-api `ops-unit` tests
Description Of Changes
Splits out execution of ops unit tests under tests/ops/api
from other ops unit tests, so they can run in parallel in CI.
This felt like low-hanging fruit to shave off some of our CI runtime for backend checks. BE checks took a total of 25 minutes with these changes, as opposed to around ~35 minutes without it, (e.g. https://github.com/ethyca/fides/actions/runs/7803890822).
As of now, it seems like our /api
and non-/api
unit tests have similar total runtimes. Unclear whether that will stay the same over time; if it becomes skewed, this parallelization will be less effective and we'd want to switch approaches. In any case, this is not a comprehensive solution to relatively slow test runtimes -- we can and should continue to explore more ways to improve our test runtimes. Namely, this particular change won't have any impact on test runtimes when running isolated/targeted tests locally.
Code Changes
- [x] support
pytest(ops-unit-api)
andpytest(ops-unit-non-api)
nox sessions, which allow for parallelization of test execution in CI
Steps to Confirm
- [x] CI still runs to completion, significantly faster. we still have all the same test coverage.
Pre-Merge Checklist
- [x] All CI Pipelines Succeeded
- Documentation:
- [ ] documentation complete, PR opened in fidesdocs
- [ ] documentation issue created in fidesdocs
- [ ] Issue Requirements are Met
- [ ] Relevant Follow-Up Issues Created
- [ ] Update
CHANGELOG.md
- [ ] For API changes, the Postman collection has been updated
The latest updates on your projects. Learn more about Vercel for Git ↗︎
Name | Status | Preview | Comments | Updated (UTC) |
---|---|---|---|---|
fides-plus-nightly | ✅ Ready (Inspect) | Visit Preview | 💬 Add feedback | Mar 21, 2024 5:19pm |
Passing run #6809 ↗︎
![]() |
![]() |
![]() |
![]() |
![]() |
⚠️ You've recorded test results over your free plan limit. Upgrade your plan to view test results. |
Details:
Merge 224380296b033d12dccdca093f9a4dfdd47d4bc2 into d0988077413d957a432c22735578... | |||
Project: fides | Commit: d8a181edd9 ℹ️ |
||
Status: Passed | Duration: 00:37 💡 | ||
Started: Mar 21, 2024 5:27 PM | Ended: Mar 21, 2024 5:28 PM |
Review all test suite changes for PR #4595 ↗︎
this could probably use a bit of polish, but wanted to at least first make sure no one saw any red flags with this incremental update!
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 86.64%. Comparing base (
d098807
) to head (2243802
).
Additional details and impacted files
@@ Coverage Diff @@
## main #4595 +/- ##
===========================================
+ Coverage 66.33% 86.64% +20.30%
===========================================
Files 338 338
Lines 19993 19993
Branches 2555 2555
===========================================
+ Hits 13263 17323 +4060
+ Misses 6257 2201 -4056
+ Partials 473 469 -4
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.