INGInious icon indicating copy to clipboard operation
INGInious copied to clipboard

[utils/task-tester] Add task tester

Open nrybowski opened this issue 3 years ago • 1 comments

Add the task tester feature. It allows replaying old submissions to ensure that tasks are still valid / functional e.g. after a modification of a task or the update of its environment.

This PR also contains a submission anonymizer. This is a script removing all the fields allowing the identification of the original submission author.

See https://github.com/UCL-INGI/.github/pull/1 for automated deployement in GitHub actions.

Steps

Task tester

  • [x] Replay submissions
    • [x] From a single task
    • [x] From a subset of the course's tasks
    • [x] From all the course's tasks
  • [x] Load additional task types from plugins
  • [ ] Improve logs
  • [ ] Compare stdout and stderr outputs and propose improvements to the task developers without making the tests fail
  • [ ] Add thread pool to parallelize tasks and jobs rerun

Submissions anonymizer

  • [x] Anonymize a set of submission files from a .tar.gz archive
  • [x] Put the anonymized submissions directly in the course directory
  • [ ] Add tests
  • [ ] Fix paths for windows
  • [ ] Handle case of single submission in the archive

nrybowski avatar Sep 08 '22 11:09 nrybowski

As the remaining tasks in this PR are mainly improvements and do not impact the functionnal requirement of this feature, I mark it as ready for review. Those future improvements will be added in their own feature requests.

nrybowski avatar Sep 22 '22 14:09 nrybowski

The logs refactor should still be done before merging the tools. This PR goes back in the draft state until it is done.

nrybowski avatar Oct 05 '22 15:10 nrybowski

Rebased on https://github.com/UCL-INGI/INGInious/commit/0959be6d8127e5a867a3e190423897069bf2eae9.

Tested on https://github.com/UCL-INGI/INGInious/pull/866/commits/9b1bf3c62d82b243e10ab006112b3a039a35eda8.

The PR is ready for merge.

nrybowski avatar Nov 03 '22 10:11 nrybowski