INGInious
INGInious copied to clipboard
[utils/task-tester] Add task tester
Add the task tester feature. It allows replaying old submissions to ensure that tasks are still valid / functional e.g. after a modification of a task or the update of its environment.
This PR also contains a submission anonymizer. This is a script removing all the fields allowing the identification of the original submission author.
See https://github.com/UCL-INGI/.github/pull/1 for automated deployement in GitHub actions.
Steps
Task tester
- [x] Replay submissions
- [x] From a single task
- [x] From a subset of the course's tasks
- [x] From all the course's tasks
- [x] Load additional task types from plugins
- [ ] Improve logs
- [ ] Compare stdout and stderr outputs and propose improvements to the task developers without making the tests fail
- [ ] Add thread pool to parallelize tasks and jobs rerun
Submissions anonymizer
- [x] Anonymize a set of submission files from a .tar.gz archive
- [x] Put the anonymized submissions directly in the course directory
- [ ] Add tests
- [ ] Fix paths for windows
- [ ] Handle case of single submission in the archive
As the remaining tasks in this PR are mainly improvements and do not impact the functionnal requirement of this feature, I mark it as ready for review. Those future improvements will be added in their own feature requests.
The logs refactor should still be done before merging the tools. This PR goes back in the draft state until it is done.
Rebased on https://github.com/UCL-INGI/INGInious/commit/0959be6d8127e5a867a3e190423897069bf2eae9.
Tested on https://github.com/UCL-INGI/INGInious/pull/866/commits/9b1bf3c62d82b243e10ab006112b3a039a35eda8.
The PR is ready for merge.