BUG: temporary remapping files are not cleaned immediately after use.

[jhamman]

I've been using the RVIC parameters generation tool to generate parameters on a 10km grid in the Arctic. The temporary remap files are pretty large (16GB each) and are not being cleaned fast enough to save enough disk space for the next file to be able to fit on the disk (200 GB in total).

For large domains with many outlets, we may need to rethink the workflow when using REMAP=TRUE and multiprocessing. For now, the work around seems that we need to use less processors so the total temporary file load remains less than the disk space.

[lizaclark]

I'm using cdo v. 1.7.1rc1 for my command line cdo, and I find that when I run cdo remapcon,file1.nc file2.nc, I get *** Error in cdo: malloc(): memory corruption: 0x0000000002e5f6d0 ***; however, if I use a grid file, I am able to get remapcon to finish. That workflow looks like:

cdo griddes file1.nc > gridfile
cdo remapcon,gridfile file2.nc

Perhaps adding an intermediate griddes step would allow RVIC to work on the temporary remap files.

[bartnijssen]

Is that a known bug in newer versions of cdo? May be good to file a bug report with them as well.