Error: std::bad_alloc when using writeCDF

[natiguvi]

I am reading a large number of ASCCI files from four years of daily data, and packing it on an SpatRaster, then I'm assing it dates and saving it as an .nc file using writeCDF, but I get:

"Error: std::bad_alloc".

I already did this about two months ago for rasters of the same size but with more years and it worked perfectly. Now when I run 4 years, I get the mentioned error.

If I run 3 years I get this one:

"Error en (function (cond) : error in evaluating the argument 'x' in selecting a method for function 'crs': std::bad_alloc",

And if I run 2 years it seems to work but I don't think is a memory problem because this is the output from mem_info(writeCDF(temp_rasters3, name, overwrite=TRUE,compression=9)):

"------------------------ Memory (GB) "------------------------ check threshold : 1 (memmin) available : 139.71 allowed (95%) : 132.72 needed (n=1) : 16.36 "------------------------ proc in memory : TRUE nr chunks : 1 ------------------------"

I was trying to attach the temp raster made when reading ASCCI files but is too big (10gb), I'm not attaching code because ASCCI files are bigger and I'm only reading it, changing crs, and packaging. The errors mentioned are when trying to save it as .nc, as raster there's no problem:

writeRaster(temp_rasters3, "temp_rasters3.tif", overwrite=TRUE) filename : D:/temp/temp_rasters3.tif compute stats : 1, GDAL: 0, minmax: 0, approx: 1 driver : GTiff disk available: 417.2 GB disk needed : 16.4 GB memory avail. : 176.8 GB memory allow. : 167.96 GB memory needed : 65.466 GB (2 copies) in memory : true block size : 1238 rows n blocks : 100 pb : 3

I'm using Windows 11 Pro 64, this is the output of packageVersion("terra")(I'm using dev version) and of terra::gdal(lib="all"):

packageVersion("terra") [1] ‘1.7.81’

terra::gdal(lib="all") gdal proj geos "3.8.4" "9.3.1" "3.12.1"

These are the options for Terra, but I also tried the default:

terraOptions(memfrac=0.95, tempdir = "D:/Temp", verbose=T, print=TRUE,steps = 100)

Thank you in advance, I look forward to your reply.

[oshuwilson]

I've been having a similar issue despite running the code on an HPC cluster with more than enough RAM

[rhijmans]

@natiguvi: It looks like that you have set terraOptions(memfrac=.9). That is much too high. Rather set it to a lower value if problems like this arise

@oshuwilson: on a HPC, you may have shared memory and I would set memfrac to a lower value, and perhaps also set memmax to a conservative value.

Unfortunately, there is not much more I can do as I cannot reproduce this.