Autograding

Open andrewbredow opened this issue 4 years ago • 1 comments

The initial plan for this will be two parts:

[x] Automated setup and feedback
[x] Automated test runs

The initial target will use GitHub Actions and will involve a custom workflow.

Aug 26 '19 18:08 andrewbredow

Autofeedback Example

The autofeedback example consists of three parts:

A template repository for the assignment which includes:
- The assignment README
- Automated tests for the assignment
- A check workflow for running the automated tests
- A setup workflow for setting up the template when used
A setup action that:
- Creates a new commit (using template files)
- Creates a branch for the new commit
- Opens a pull request (using the PULL_REQUEST.md template)
A student repository (built from the template)
- On each new push automated tests are run

Template repository

Here's a quick explanation of all of the files:

File	Description
`README.md`	Contains a description of the assignment, instructions for getting started and links to the pull request
`add.test.js`	An automated test that will pass or fail based on the code the student writes
`jest.config.js`	The automated test configuration
`package.json`	The package.json in this case is only used for running the tests and installing the dependencies for this assignment (in this case there are no dependencies other than Jest for running the test)
`package-lock.json`	see `package.json`
`.gitignore`	General `.gitignore`, most importantly there are entries for `node_modules` and `add.js` The entry for `add.js` is removed when setting up the template (see `.github/template/files/.gitignore` below)
`.github/workflows/check.yml`	This workflow runs the automated tests
`.github/workflows/setup.yml`	This workflow only runs once, on the first push of the newly created repository. It automatically generates a commit, a new branch, and a pull request for that commit so that students can begin working immediately
`.github/template/PULL_REQUEST.md`	The template for the pull request that will be opened. This let's teachers get as creative as they want
`.github/template/files/add.js`	All of the files in this `files` directory (including `add.js` will be used to create the commit for the pull request branch.
`.github/template/files/.gitignore`	This .gitignore is the same as the main .gitignore except it allows `add.js` to be added to the repo.

There are a lot of pieces for creating a template here. It would probably be helpful to put together a generic "assignment" template repository for teachers (in different languages).

Try it out as a student

To get the student experience (short of creating a full assignment and accepting it) follow these steps:

Go to the addition assignment example and click Use this template.
Name the repository example-addition-assignment then click Create repository from template