otter-grader icon indicating copy to clipboard operation
otter-grader copied to clipboard

Add manually graded questions within the same gradescope assignment

Open nadia-eecs opened this issue 10 months ago • 2 comments

Description

Add manually graded questions within the same gradescope assignment. For example, when configuring Gradescope programming assignments (without otter-grader) rich formatting can be added to the autograder output, where there is a "html" option. This means that manual test cases can be treated as "autograder" (in the otter grader sense) to generate the associated .py test files. The files can then be ran to plot student functions and write them to html by creating a temporary file/BytesIO object. By creating a function that runs student plots the images can be rendered in the output of the same assignment. Below is an example of what the feature could potentially look like (source does not utilize otter-grader or notebook homeworks but standalone functions)

Screenshot from 2024-04-30 18-26-22

Implementation Strategies

Strategy 1:

Create a custom plugin that configures Gradescope autograder with html output format

Strategy 2 (compliments of @chrispyles)

  1. add a new key to the AutograderConfig to enable this behavior (something like export_manual_responses ?).
  2. update otter assign to somehow link manually graded portions of notebooks to their question name.
  3. add a method to the GradingResults object that accepts a notebook, extracts all of the manual response cells, and stores them keyed by the question name
  4. update GradingResults.to_gradescope_dict to convert the manual cells to Gradescope’s format and include them in the results dictionary it generates
  5. update both the python and r runners run methods to call the new GradingResults method with the submission notebook (note that it’s possible to submit non-notebook files which also get run by these runners, so this step should only be performed if the submission file’s extension is .ipynb)
  6. add unit tests for all new behavior and update the changelog

Feature advantages

The feature would enable students to submit one notebook file with all test cases manual and automatic outputs written to Gradescope autograder directly allowing to streamline the grading process for larger undergraduate courses reliant on python notebook submissions.

nadia-eecs avatar May 01 '24 03:05 nadia-eecs