Failed writing body

[CuddleFish18]

I have a question about an error I receive. On the internet I found some answers about the size of my server. However, I run the code on an AWS server with 488 GB RAM. The Rda file I try to load is 96 GB. So I don't think that is the problem. When I use s3fs to mount the S3 bucket I can load the file in to R with

load("s3mnt_folder/folder_xxxx/file_xxxx.Rda")

Loading the raw file works just fine:

df <- get_object("s3://bucket_xxxx/folder_xxxx/file_xxxx.Rda")

Here is my code:

## load package
require(aws.s3)

## code goes here
> Sys.setenv("AWS_ACCESS_KEY_ID" = "xxxx",
+            "AWS_SECRET_ACCESS_KEY" = "xxxx",
+            "AWS_DEFAULT_REGION" = "region_xxxx")
> 
> e <- new.env()
> s3load("folder_xxxx/file_xxxx.Rda", bucket = "bucket_xxxx", envir = e)
Error in curl::curl_fetch_disk(url, x$path, handle = handle) : 
  Failed writing body (2602 != 16360)

## session info for your system
> sessionInfo()
R version 3.5.0 (2018-04-23)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: Amazon Linux 2

Matrix products: default
BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] aws.s3_0.3.12

loaded via a namespace (and not attached):
 [1] httr_1.3.1          compiler_3.5.0      R6_2.2.2            tools_3.5.0         base64enc_0.1-3     curl_3.2            Rcpp_0.12.18        aws.signature_0.4.4
 [9] xml2_1.2.0          digest_0.6.16  

[MDunsker]

Did you ever find what was causing this error? I am facing a similar issue with a "failed writing body" error in curl.

[JanLauGe]

For posteriority: I've encountered this error when there was no disk space left to write the temp file.