Submission via Git-Repository

[sargreal]

In advanced programming courses it is quite often the case that the development in groups would happen with git. It is possible to submit these to INGInious via the file problem-type but this seems quite unintuitive and is also not realy advised (see best practices). Using code problems is even worse because then for large projects the student needs to copy paste every single file and if the student want's to create new files it ist not even possible.

So it would be really practical to integrate git completely into INGInious for a better workflow for both students and instructors.

• The easiest way to impement this would be with a problem type that has inputs for the repository url, git reference (Branch, tag or commit sha) and the credentials (optimally with access token rather than password). The instructor could also provide a base url and testing repository. This approach would require the cloning of the repository manually in the run script.

• Better would be to extend/modify the Git Repo plugin by not (just) saving the archives to the repository but pulling the latest changes from the provided repository url and then mounting that into the container. The mounting would probably not work when the docker-agent is not running on the same machine as the webapp. Therefore it might be necessary to still submit the cloned repository as archive and then decompressing it in the container automatically by the INGInious script or manually in the run-script.

• An even better solution in terms of would be to integrate git into the common module and let every component connect to it just by submitting the submission id and the database having stored the commit sha of the latest commit on submission. Then it could get the respective repository data and send it as reply to the component. I think this would however be the most complicated option and would make the integration a major component instead of a plugin.

For reference other tools like Submitty or Artemis already use full git integration or even use git as the base option for saving and submitting code. These tools however seem to me ore complicated and are much more complex at setup than inginious. Also the user interface for Inginious is much more intuitive. It would therefore be very nice if git integration could become a full feature of inginious!

[erelsgl]

I agree. In our university, we had to build a special tool only for handling github submissions: https://github.com/SamuelBismuth/badkan but it has other issues. It would be better to have this functionality in inginious.

[javawolfpack]

I just asked about multiple files even seeing that it is not in the best practices. For us beyond and sometimes even in our CS1 course, we have .cpp & .h files, and in most cases, multiple files the students are authoring for a project. It would be nice if git integration were an option/alternative, especially if that is considered best practice.

I'm trying to run INGInious as an option to provide auto-grading and not pass the cost to our students through tools like zyLabs/Codegrade. Both support file upload and Codegrade has git integration. For the same reason, most projects beyond an intro course are usually multiple files.

[anthonygego]

I wish that @augustindelecluse share his very interesting experience with working with git repositories publicly so that we can make progress on this discussion.

It would be great to end with a common flow that would match most of the needs. Storing the repository at the INGInious side is currently not possible given the current architecture. Fetching the code is more easy and the probably easiest way to is to perform the operations like they are done with a CI: letting the students add a deploy key to their repository. Then, they submit the repository URL, possibly with a branch and hash. But I know @augustindelecluse has some more complex workflow.

Currently those steps are to be done manually in the run file but they could probably be automated in a specific subproblem type that would take those inputs and settings (or even generate the deploy key itself) and return a tarball of the repository files when get_input <problemid> is invoked. I don't know if such a feature would currently be useful to someone.

This would however add load to the frontend, which currently does not support any kind of asynchronous operations, making one request 'slot' at the webserver side unavailable during the repository processing time.

[augustindelecluse]

Thanks @anthonygego for the notice. We use git for several courses, such that we can pull and evaluate a repository containing multiple files. In practice we use

1. A solution repository, where the teacher writes expected solution for the course / project (exemple here).
2. A template repository automatically updated from the solution using github action. The solutions written in the solution repository are delimited by specific tokens. The parts between those tokens, that the students have to implement, are removed in the template, leaving only a skeleton waiting to be filled in with the student implementation (exemple here).
3. Several student repositories that are automatically created from the template repository.

This is in turn is used with inginious with 2 types of tasks:

1. The creation of a student repository. The student must submit a github username used for the creation of the repository. (exemple here)
2. The grading of a student repository. There is no submission field: we retrieve the repository based on the inginious username of the student, clone it, run several checks to ensure that it compiles and is up to date with respect to the template, and finaly test it (exemple here).

All of this is done using github cli, bash and python within run files. However, creating a new grading exercise is still easily configurable: all the important pieces are written within a json file, where information about the file to grade, files to archives are written. The creation of a grading exercise is done in a matter of minutes. The version that we use is aimed for the grading of java tasks, but could be easily adapted for other languages.

If this workflow could be used for your courses as well, I can create a github repository containing the run files for the tasks as well as documentation for the setup, so that you can play with them. Alternatively, we could adapt it to create a git task.

[javawolfpack]

Didn't occur to me to potentially feed text input to a run script. Assume we could have our run script and then just clone/run the code, likely error-checking for repos that are not organized correctly or don't compile. Maybe feed public/private keys in somehow? Or students provide an authentication token & username.

I know all of my courses I have a web tool that generates repos in an organization for the class, so could make a TA/grader account that has read access to everything that could just be used to clone in the run script.

I'll give it a try when I finally get our instance up, pending SAML/Shibboleth details from my campus IT.

[augustindelecluse]

If your goal is to clone and run the code from a repository, I think the best option is to have a task taking as input an ssh link to a repository as well as an authentication token. The task would first clone the repo, and then run the tests for it.

I assume that your web tool generates repositories based on the students usernames. For instance student A would have a repository "project-A" and student B "project-B". If you have a grader having read access to the repository within the task, asking for the ssh link without an authentication token is not secure enough. Nothing prevents student A from feeding as input a link to "project-B", and being graded based on student B's work. The token is necessary to be sure that the repository belongs to student A.

In the tasks examples that I provided above, we get around this problem by mapping the inginious usernames to repositories within the run scripts. The students never provide any link to a repository, and there is no need to check that the repository evaluated is effectively their own - it is the case by construction. From the students perspective, they do not need to copy paste any ssh link or token for the submission of task: there is no input required for the grading tasks. This makes the tasks more user friendly