Improve shiny prerendered process by avoiding race condition when writing file

[cderv]

I have a suspicion that there are somehow two processes running at the same time to prerender the source document and are inadvertently writing into the same file. I've uncovered a reproducible example that produces the expected error (here shown before the fix in #2500).

render_bg <- function(path) {
  callr::r_bg(
    function(path) {
      rmarkdown::render(path)
    },
    args = list(path = path)
  )
}

# Remove prerendered HTML and source if they exist
unlink("issue.html")
unlink("issue.Rmd")

writeLines('---
title: "issue"
output: html_document
runtime: shiny_prerendered
---

```{r}
textInput("text", "Text", "Text")
renderText(input$text)
```', "issue.Rmd"
)

# Render the Rmd twice at basically the same time
r1 <- render_bg("issue.Rmd")
r2 <- render_bg("issue.Rmd")
r2$wait()

When you try to run the document, it will have been corrupted by simultaneous writes into the .html file.

rmarkdown::run("issue.Rmd")
#> Error: parse error: trailing garbage
#>           ":{},"value":["2.23.3"]}]}]} {"type":"list","attributes":{},
#>                      (right here) ------^

After applying the fix in #2500, we still get errors due to failures in other places in the file.

rmarkdown::run("issue.Rmd")
#> Error: lexical error: invalid char in json text.
#>           s":{},"value":["jquerylib"]} <!--html_preserve--> {"type":"l
#>                      (right here) ------^

Originally posted by @gadenbuie in https://github.com/rstudio/rmarkdown/issues/2499#issuecomment-1651927964

In this case, my reprex is a hypothetical example that's at least consistent with the real-world scenario. I've heard reports of this kind of problem for learnr users on Connect and shinyapps.io as well as Posit Cloud. Cloud is surprising to me because in theory it should be equivalent to a local installation, where this problem hasn't been reported really.

Related to the concurrency option, I understand that there are many reasons that make writing unique files difficult. That said, if we're using a mechanism like writeLines(text, file_connection), I do think rmarkdown/knitr should write into a temporary uniquely-named file that's moved into place when the contents are finished. This way, even if there are race conditions, then at least the final intermediate file is the atomically the complete and correct winner of the race.

Originally posted by @gadenbuie in https://github.com/rstudio/rmarkdown/issues/2499#issuecomment-1651998095