continuous_evaluation
continuous_evaluation copied to clipboard
TODO of CE restructure.
DB:
- [ ] Deploy MySQL DB, for example, XDB in baidu.
- [ ] Design DB schema.
- [ ] Familar with DB client API.
Monitor
- [ ] Monitor tasks' status and send warning email if a commit's task fails.
- Don't send warning email repeatedly.
- New commit's success can overlap old commit's failure.
Report
- [ ] Send daily (or weekly?) report to the email group
- [ ] Need a new email group to include developers, users, and leaders.
- [ ] accuracy, speed, memory usage
Web Monitor
- [ ] List recent commits' tasks status order by time.
- [ ] List history commits' task status by time region.
PaddleCloud
- [ ] Prepare our docker image run on paddlecloud.
- [ ] Prepare paddlecloud client to commit our docker image to run jobs.
- [ ] Give a task (env) interface.
Change of old CE:
- [ ] Add DB support
- [ ] Change shell script from sync to async and call paddlecloud client.
Future: Add more models to CE.