multi-model-server
multi-model-server copied to clipboard
CircleCI + Performance Regression
Description of changes:
Primary goal of this PR is -
- Integration of performance regression suite with CircleCI
- Add new performance regression job in circleci config
- Add corresponding shell script for job execution
- Added new ci_linux_medium environment for comparing against thresholds specific to CircleCI env.
- Renamed xlarge environment to macos_xlarge (more specific)
Along with the primary goal, we have identified and fixed following -
- Movement of API tests(newman) suite from test/ to tests/api/
- CircleCI's API tests(newman) job now continues with execution of all 3 collections(management, inference, https), even if any one of them fails.
- CircleCI's shell scripts tweaked to exit with error when any of the important commands fail.
- CircleCI Jobs now copy the entire MMS's logs directory along with console logs (instead of only copying mms console logs)
- Removed - /performance_regression. The directory was part of POC and is no longer required.
All Jobs are working and are successful after these changes -
Build #443

Performance regression Job Performance regression logs and results
Note:
The commit count is high because the previous PRs of CircleCI and Performance regression have been squashed & merged.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.