Is Arguments$getWritablePath() parallel safe?

[HenrikBengtsson]

Is Arguments$getWritablePath() really parallel safe? What about fileAccess()? I certainly had that in mind they were written, e.g. there is code for trying multiple times. This has been tested thoroughly by the Aroma Framework etc. However, it could still be that race conditions could show up, cf. https://stackoverflow.com/questions/45837562/catch-an-error-produced-by-pbsapply

[complexgenome]

Hi Henrik, I'm using REIDS package which in turn calls aroma. In this process, I get error:

Error in getWritablePathname.Arguments(static, ...) :

I'm working on PBS job scheduler. 4.9.0-4-amd64 #1 SMP Debian 4.9.51-1 (2017-09-28) x86_64 GNU/Linux R version 3.4.2

other attached packages:
 [1] REIDS_0.1.0            GenomeGraphs_1.38.0    biomaRt_2.34.2
 [4] aroma.light_3.8.0      aroma.affymetrix_3.1.1 affxparser_1.50.0
 [7] aroma.core_3.1.3       R.devices_2.16.0       R.filesets_2.12.1
[10] R.utils_2.7.0          R.oo_1.22.0            R.methodsS3_1.7.1

Can I solve it?

[complexgenome]

Hi, Was wondering if there could be a workaround for this? Thank you.

[Kodiologist]

fileAccess is indeed not parallel-safe. Here's an example using R.utils 2.12.0 and gunzip, which calls fileAccess internally:

options(warn = 1)
library(parallel)

set.seed(10)
cat(file = "test", rnorm(1e6))
R.utils::gzip("test", overwrite = T)

invisible(mclapply(mc.cores = detectCores(), 1 : 5000, function(i)
   {temp.path = tempfile(paste0("myf-", i))
    on.exit(unlink(temp.path, expand = F))
    R.utils::gunzip("test.gz", temp.path, overwrite = T, remove = F)
    Sys.sleep(runif(1, 0, 3))}))

This produces warnings like Warning in file.remove(pathname) : cannot remove file '/data/scratch/rtem/Rtmp64oOjl/w', reason 'No such file or directory' when the temporary file names chosen by different fileAccess calls collide.

Besides, the very practice of checking that the file is writable before doing the write of interest creates a race condition.