DumpExporter fills up RAM

[magula]

cmsDumpExporter will put everything in a tempdir (i.e. usually /tmp) before packing it up in an archive at the desired path: https://github.com/cms-dev/cms/blob/7d962150d9969281148d3eaba1ab35823fdfb1b6/cmscontrib/DumpExporter.py#L189-L190

On some systems, including Arch, /tmp is in RAM by default, and limited to half the RAM's size. At least on personal computers, this will usually cause the system to freeze or crash. Of course, CMS writes to /tmp on other occasions, but in normal operation, it will cleanup shortly afterwards and never write something as large as the whole database at once.

It seems that by setting one of the TMPDIR, TEMP, or TMP environment variables, one can specify another location, so circumventing the issue is technically easy. However, not everyone will know this and think of it beforehand, and a system freeze is annoying, as is finding out the cause of the problem.

Thus, I would like to suggest improving this somehow.

• The files have to be written somewhere of course, and using tempfile.mkdtemp makes sense. Easy and still helpful would be a warning, so at least one will know why the system froze (or be able to hit STRG+C if fast enough...). This should include suggesting to set TMPDIR to somewhere big enough.
• Maybe it could even explicitly ask along the lines of Path to temporary directory (default is system temp dir; sufficient free space is needed):
• Or it could use the directory where it shall export the archive. Drawback: If the exporter crashes, it won't be cleaned up.

This sounds similar to #322, but I think that was more about importing and exporting specific contests and is unrelated.

Happy to hear your thoughts!

PS: This has been in the back of my head since forever and I haven't seen someone else mention it, so who knows if I'm the only one with this problem. :smile: