Shiny app unresponsive after R session's tempdir is deleted (by Windows)

[ismirsehregal]

System details

Browser Version: Google Chrome Version 76.0.3809.87 (64-Bit)

Output of sessionInfo():

R version 3.6.0 (2019-04-26)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

Matrix products: default

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] shiny_1.3.2

loaded via a namespace (and not attached):
 [1] compiler_3.6.0  magrittr_1.5    R6_2.4.0        promises_1.0.1  later_0.8.0     htmltools_0.3.6 tools_3.6.0    
 [8] Rcpp_1.0.2      digest_0.6.20   xtable_1.8-4    httpuv_1.5.1    mime_0.7   

Example application and steps to reproduce the problem

# Windows shell required
tempdir()
dir.exists(tempdir())

library(shiny)

shinyApp(
    ui = fluidPage("Please reload to see me fail."),
    server = function(input, output) {
        shell(paste("rmdir", dQuote(
            normalizePath(tempdir(), winslash = "/", mustWork = FALSE), q = FALSE
        ), "/s /q"))
    }
)

Describe the problem in detail

I have already tried to get help via the rstudio community and by reviving this issue, where Dean Attali suggested using tempdir(check = TRUE) in shiny's source code what seems reasonable to me, regarding the following:

I have a shiny app deployed locally via RScript.exe and runApp on a Windows machine which runs just fine. However, when the app is idling for a few days the browser displays the following:

This is unrelated to the apps content.

I was able to generate a log file which gave me:

Warning in file(open = "w+") : cannot open file 'C:\Users\USER\AppData\Local\Temp\RtmpO0mnd9\Rfdd88a66def': No such file or directory Warning: Error in file: cannot open the connection [No stack trace available]

... and indeed the according folder generated on starting up the R session and the files generated on starting the shiny app were deleted (automatic cleaning done by Windows? - I'm not sure why).

I am able to reproduce this error when I'm intentionally deleting the R sessions tempdir() hosting the shiny app (as shown above).

It seems also others are affected: please see this and this.

In my eyes it would be sufficient if the shiny application exits with an error in this situation instead of continuing to run in an unresponsive manner.

Adding an argument tempDir to runApp() (defaulting to tempdir()) would also be conceivable.

---

By now I've found a setting in Windows 10 (Storage Sense) concerning the deletion of temporary files, which seems to be active by default.

Navigate as follows and uncheck:

1. Settings
2. System Storage
3. Storage Sense
4. Change how we free up space automatically
5. Delete temporary files that my apps aren't using

Nevertheless, in my eyes this situation should be prevented in the shiny source code or the app creator should have the possibility to specify where the apps temp files are stored (without setting environment variables).

[wch]

My guess (without having tested this) is that this particular error is happening here: https://github.com/rstudio/shiny/blob/178872d/R/utils.R#L1516

There's a long discussion thread on the R-devel list about this issue of temp dirs being deleted here: https://r.789695.n4.nabble.com/tempdir-may-be-deleted-during-long-running-R-session-tt4732832.html

R code in general (not just for shiny) is written with the assumption that tempdir() will return a path that exists, and tempfile() will return a path within an existing directory. So even if we used tempdir(check=TRUE) in shiny, it's likely that there will be other code in other packages that has the same problem. And also, with tempdir(check=TRUE), that only ensures that the temporary directory will be created if needed -- if the R code relies on the continuing existence of any files in the temp directory, then if the directory is deleted and re-created, that will be a problem.

I guess this is a long way of saying that the problem is with R: the docs for these functions refer to a "per-session temporary directory" which is "created before the interpreter is started", but then provides no guarantees at all about the lifetime of the temp dir or files within it. We could use tempdir(check=TRUE) and tempfile(tmpdir=tempdir(check=TRUE)) everywhere, but I'm guessing this will just lead to other weird problems down the line.

[ismirsehregal]

@wch thank you for your response!

During my research I also came across the thread on the R-devel list. I absolutly agree with this statement from Duncan Murdoch:

The problem occurs as soon as the tempdir() gets cleaned up: there could be information in temp files that gets lost at that point. So the solution should be to prevent the cleanup, not to continue on after it has occurred (as "check = TRUE" does). This follows the principle that it's better for the process to always die than to sometimes silently produce incorrect results.

However, currently a shiny app, which was launched before tempdir() is deleted, won't "die". It keeps on running and displays "ERROR: cannot open the connection" when new sessions are requested. I'd prefer it to crash in this scenario.

Of course, the best way is to avoid the deletion of tempdir() in the first place (which I did on my system by now). However, it took me a long time to find out what is going wrong in the shiny context. That's why I think a more sophisticated error message and/or adding an argument tempDir to runApp() (defaulting to tempdir()) with a proper description of the use case or the problems which may arise using tempdir() would be quite helpful.

You definitely have a point regarding tempdir(check=TRUE), as we cannot restore what is lost already. However, using tempdir(check=TRUE) in the shiny source code has the advantage, that even though we aren't able to revive a running app, we can at least launch a new app in an existing R session (which isn't possible without check=TRUE). Please see the following:

Another possibility to prevent the relevant files from beeing cleaned automatically would be to touch them (change mod. date) every x hours - but this also seems to be a makeshift solution.

[mingsnu]

Similar issue here. My teammates scheduled a cron job to clear the tmp folder periodically in the linux host for certain reason and it breaks my shiny app. Is it possible to add a parameter to let the user specify the tempdir location? Thanks!

[ismirsehregal]

Here are some options described to change the location of tempdir().