russross
Screenshots?
Hello, this seems really cool! I was curious if you had any screenshots of the web interface, when it's embedded in canvas? I am looking for something like this that could be used for TypeScript, Octave, Java, and a few other languages. We've been using VPL embedded in Moodle, but I'd rather find an LTI canvas option. It seems like making new docker files for other languages is pretty easy? I was also curious what the "Report Card" data from the server looks like. I don't suppose there a demo server running anywhere?
We actually moved away from the web interface. All that gets embedded in Canvas is some setup instructions. Instead we have two UIs:
- A CLI tool called "grind", which is used by students and also instructor-only extensions for creating problems and interacting with student code
- A Thonny plugin (targeted at CS1/CS2 students) with a subset of the functionality of the grind tool integrated into the Thonny menu system
I recorded a screencast last year with a brief demo for my instructors that gives the basic feel of both tools:
https://www.youtube.com/watch?v=v-PA_LvlEoY
A few things have changed since then, most notably the distinction between subdirectories and files in the main directory is no longer there.
One of my design goals with CodeGrinder was to expose students to native environments and tools whenever possible, hence the move away from a web interface. We use it in a few upper-division classes as well through the CLI, and I try to make a point to my students about all the tools being there on their local machine. You can run everything through the Makefile, and CodeGrinder just does it for you in a controlled environment.
Adding new languages is pretty easy, but it is a manual process (creating directories and support files directly in the distribution, writing a Dockerfile, adding a few entries to the database). For some languages I create one problem type based on unit tests and another based on feeding them input and comparing the output to a reference.
The report card is just a transcript of the code running in the Docker container and the final score they received. It is posted to Canvas whenever they run the grade command so you can see it (along with their source files) in the SpeedGrader interface.
I'm happy to answer any other questions if you would like to pursue it further. It's pretty simple to set up a server if you have Go and Docker both up and running. I mostly run mine on a couple Raspberry Pis. The installation instructions in the README are reasonably up-to-date, but it is missing a HOWTO on creating a problem and taking it through the lifecycle as well as any description of the structure of problems and problem sets.
One more screencast:
https://www.youtube.com/watch?v=mz2S5z8F3j8
This one was to introduce CodeGrinder to other instructors (but not administrators). It walks through setting up the external app in Canvas, creating simple problems through the grind tool, and assigning those problems in Canvas.
Very interesting! I'm still interested in it, though I'd have to setup some kind of very thin web frontend for our purposes. I'm thinking literally just a Run button and a CodeMirror editor, really. I'll have to play with that.
For now, I am still getting it set up on our server. Is Let's Encrypt a hard requirement? It looks like the server depends on it for the HTTPS certificate. For dev purposes, I was hoping to just use a self-signed one. But I think I'd have to swap out these lines here for some other Go code?
For my dev server I assigned it a name and just use Let's Encrypt, but you could remove that pretty easily. Here's a code example that might help: https://gist.github.com/denji/12b3a568f092ab951456#tls-transport-layer-security--server
Making a web client is pretty straightforward. All requests are HTTPS+JSON or Websockets+JSON and the API is pretty straightforward. If you are unfamiliar with Go there are a few things that might help:
- The Thonny plugin has Python 3 implementations of the main student workflows.
- The grind client can report on every server call it makes. You need the .codegrinderinstructor features enabled (see the screencast) and then just add --api to any grind command and it will echo the server requests it is making. Use --api-dump and it will dump the request and response objects to the console.
A couple other concepts that will help make sense of the workflows:
- The TA server and the Daycare server(s) do not communicate directly (other than when daycare servers register themselves with the TA). Daycare servers are stateless and disposable and you can have as many as you want. The TA does not have any long-running requests, so it is pretty resource-lite and should be able to handle any reasonable number of students.
- To grade a problem step, the client sends a commit bundle (bundles are requests with multiple kinds of objects) to the TA, which saves it, adds some supporting objects, signs it, assigns a daycare server, and sends it all back to the client. The client then opens a websocket to the assigned daycare, submits the bundle, and then streams data back and forth over the websocket as the code is tested in a Docker container. The last message from the daycare contains a new signed commit bundle with the reportcard attached. The client submits the updated bundle to the TA, which saves it and reports the grade back to Canvas.
- Each problem type defines a series of actions (mirrored by rules in the Makefile) that a student can run. They can generally run tests without reporting a grade back, you can run lint tools (and modified files can be posted back to the student), and you can even run interactive sessions with a TTY to run the debugger, etc. The interactive sessions may have fallen into disrepair a bit since I push my students to do everything locally as much as I can.
I'll stop there before this gets too long, but I'm happy to answer more questions. If you think a Zoom meeting would be helpful to go over questions and demos I'm open to that as well.