Homework module

[maxlieblich]

Here are a few ideas for a homework module. I will suggest a workflow for certain aspects of this soon. Maybe this should be multiple issues. Forgive me; I am a github newbie.

Problems

• There should be a bank of problems.
• Problems should be optionally written in a high-level DSL created to be user friendly, but there should be an API for creating problems in any language (e.g., Clojure, Java, Lisp, Python, Perl, LaTeX, ...)
• Problems should be tagged. Fantasy: the DSL could be structured to automatically give any contributed problem (semantically appropriate) tags.
• The problem bank should be easily and quickly searchable.
• Each problem should have associated (suitably anonymized) performance data, coded in such a way that an instructor can access the data for just his or her class, for his or her school, city, country, etc. These performance data will be crucial for making adaptive assignments.
• There should be problems that take LaTeX or Markdown answers and require human grading (or some kind of sophisticated future "unnatural language" AI grading) in addition to the usual problems that can be graded by computer.

Problem sets

• There should be a simple and fast problem set creation tool for assembling and scheduling problem sets.
• There should be an automatic problem set creation tool that will create a problem set with a mix of problem topic tags and difficulty levels (as determined by performance or by writer-rating)
• There should be an API for hooking up a machine learning model to make adaptive homework. For example, give a list of tags and a maximum problem set length, and use the historical performance data to model the current student's performance and decide on which problems will fix weaknesses in the student's knowledge.

Analytics

• Performance data should easily enable problem set grading in multiple ways: typical percentage grades, student clustering, etc. Perhaps a few chunks of scikit-learn can be wired up to the data to make a simple setup for common models -- HMMs, clustering, ....
• Metrics decided upon by the instructor should be easily exportable to CSV (or other format?) for importation into other gradebooks.

First step: establish a workflow for problem creation, storage, and search, and a suitable data structure for problem performance (right/wrong, time taken, number of attempts, etc.).

[haraldschilly]

Hi, thanks for all the thoughts. First, all this might be better suited in the wiki as a design document. Over here, it's more like a linear flow of discussions and that is getting confusing very soon. Instead of * points, I would also suggest you to always write 1. to number it - that makes referencing easier.

We could also use SMC itself to have a shared project and write up design documents there.

ad DSL: I'm a bit scared to have a DSL for this. Rather, I would prefer to settle on a very simple data-format; personally I would like to see this happen in YAML. I think that covers all what you need. It's easy to make it programmable in all the languages you mentioned, too. (well, except LaTeX, I don't understand why you would want to program in TeX)

ad Analytics/1: Yes, that could be really interesting to do, in several aspects.

[maxlieblich]

Thanks for your comment. I will put something in the wiki. (Regarding LaTeX: I'm just thinking of people who might want to contribute problems who aren't programmers but do write math documents.) YAML is a good idea!