ok-client
ok-client copied to clipboard
Backups should not be collected if the code has not changed
Students like to run the tests multiple times in a row. The extra backups take up space and bandwidth, and it's more noise when looking for a backup to restore.
This could be implemented by taking a hash of all file contents (preferred), or storing file modification times.
The fact that they attempted another test run is logged in the analytics protocol - so we don't really need their backup to count attempts.
A small tweak would be to just not send file_contents if there are no changes (and log that there were no changes). There would have to some minor changes to server to not display/use these backups.
To avoid extra network traffic, we could bundle these "duplicate" backups up in .ok_messages until theres is an actual backup/submission to send.
It's a bit more hassle (compared to ignoring backups with no modifications), but I think it's worth it.
- Unlocking suites often may involve no changes to the code, but we still want those sent to the server at some point for analytics/research
- A student might not make any changes but is now requesting a hint, we want that info for analytics/research
The key part of the above cases is that it's not critical to have those immediately sent up - so bundling backups with no modifications to be sent later (without the file_contents message) seems like a good idea.
I didn't realize that we'd still want analytics messages. This seems nontrivial and we could do without it, so I'll remove the Fall 2016 milestone for now.