Check for low disk space
The server should check for low disk space before storing files. I don't know if this is the case already but I had problem with this. I didn't looked the server for a while and the storage went full. During this time the server could not compile the submissions and the database got corrupt (The executables from these submission are showed in the database but they don't exist in the disk)
I think the most problematic service here is the worker, as its cache stores all the executables and never deletes them.
I don't think this should be handled by CMS itself. Contest administrators should monitor disk space on their machines with appropriate tools (e.g., Icinga).
I agree that monitoring should not be done by CMS, although having a way to either limit the size of the cache and/or to trigger some form of automated cleanup (smarter than just a rm -rf) would probably be nice.