`rmarkdown::render` does not respect symlinks

[SamDM]

When rendering a .Rmd file that is a symlink, the working directory becomes the parent directory of the original file where the symlink points to. As a result, the output appears in a different directory than the .Rmd file.

See the example below:

mkdir /tmp/foo
cd /tmp/foo
mkdir bar
mkdir baz
cat > bar/example.Rmd
---
title: Test
output: html_document
---

```{r}
getwd()
```
^D
ln -s ../bar/example.Rmd baz

The file structure looks like this:

/tmp/foo
├── bar
│   └── example.Rmd
└── baz
    └── example.Rmd -> ../bar/example.Rmd

Now I proceed to rendering the html:

cd baz
R -e "rmarkdown::render('example.Rmd',output_file='output.html')"

Now the file tree looks like this:

/tmp/foo
├── bar
│   ├── example.Rmd
│   └── output.html
└── baz
    └── example.Rmd -> ../bar/example.Rmd

Also, the output of getwd() in the html equals "/tmp/foo/bar"

In other words, running the command for rendering /tmp/foo/baz/example.Rmd resulted in /tmp/foo/bar/example.Rmd being rendered instead. That is, render followed the symlink before rendering the document.

The expected behavior would be that markdown::render treats symlinks like normal files and does not follow them. At least, the current docs do not mention anything about this special treatment for symlinks.

I'm not complaining though, rmarkdown is pretty neat. I just bumped into this and wanted to point it out :)

[earcanal]

I ran into this today, and agree that it should render into the same location as the symlink.

[yihui]

My guess is we called normalizePath() somewhere in render() to resolve the symlink. I don't have time to dig into it. Sorry.

[jherrero]

Same here. In case this helps anyone, I got around by adding output_dir = "." as in: R -e "rmarkdown::render('example.Rmd',output_dir='.',output_file='output.html')"

Not perfect, but it works.

[gorgitko]

@jherrero This still doesn't work - although output.html is created in the current working directory, the example.Rmd is executed in its source location (try to output getwd() in some chunk).

My current workaround is following:

rmd_tempfile <- tempfile(tmpdir = ".", fileext = ".Rmd")
file.copy("example.Rmd", rmd_tempfile)
rmarkdown::render(rmd_tempfile, output_dir = ".", output_file = "output.html")
file.remove(rmd_tempfile)

tl;dr Create a temporary copy of the source .Rmd in the current working directory.

[cderv]

@SamDM based on your example, as @yihui said, R will follow simlink when normalizing the PATH.

$ cd /tmp/foo/baz
$ Rscript -e 'normalizePath("example.Rmd")'
[1] "/tmp/foo/bar/example.Rmd"

and we call it in render() on the input file : https://github.com/rstudio/rmarkdown/blob/69b54b68a586a59e0d8fc678e989e69eb8b3d376/R/render.R#L349

Also we execute the code in the input directory https://github.com/rstudio/rmarkdown/blob/69b54b68a586a59e0d8fc678e989e69eb8b3d376/R/render.R#L385-L387

so it will also be in foo/bar not foo/baz

Setting the output_dir="." will output in the correct folder but the default working dir will be the one from the normalized path. knit_root_dir in render could solve that.

rmarkdown::render('example.Rmd',output_dir='.',output_file='output.html', knit_root_dir = normalizePath('.'))

it requires a normalizePath because it isn't normalized inside render()

So yes, this is an issue currently. rmarkdown does not work well with symlink currently.

Maybe using fs to handle file paths in rmarkdown could help with all the paths manipulation. It would be the correct path if we want absolute one:

$ Rscript -e 'fs::path_abs("./example.Rmd")'
/tmp/foo/baz/example.Rmd

However, changing anything like this in rmarkdown could lead to side effects... 🤔

[jmw86069]

Adding +1 to the request to handle symlinks as proper local files, and not retreat back to the source file and source file name.

It is fairly cumbersome to override all the parts of rmarkdown::render():

• knit_root_dir
• output_dir
• intermediates_dir

rmarkdown::render("filename.rmd", knit_root_dir=x, output_dir=x, intermediates_dir=x)

Even then, the cache uses a subdirectory of the source file. I end up copying the .rmd to each folder where I want to run, and I can only run once per folder, even when using parameters.

These variables are not defined in RStudio - so even though RStudio runs the symlinked .rmd file, it always saves the output in the directory with the source file. Therefore the viewer does not open.

I tentatively suggest a straightforward workaround that handles the file path independent from the .rmd file.

The one-liner replacement is shown below:

rmd <- file.path(normalizePath(dirname(rmd)), basename(rmd));

The idea is to normalize the directory and not the file.

Some examples to illustrate what is happening:

# get just the .rmd filename
rmd_basename <- basename(rmd)
# get the path for the .rmd filename
rmd_dirname <- dirname(rmd)
# normalize the path independently
rmd_dirname_norm <- normalizePath(rmd_dirname)
# assemble normalized path, and 
rmd_absolute <- file.path(rmd_dirname_norm, basename(rmd))

# testing several iterations for rmd are all TRUE
file.create("test.rmd")
dir.create("testing_rmd")
file.symlink(
   from="../test.rmd",
   to="testing_rmd/renamed.rmd")
rmd_tests <- c(
   "test.rmd",
   "testing_rmd/../test.rmd",
   "testing_rmd/renamed.rmd")
for (rmd in rmd_tests) {
   cat(paste0(" input rmd file: ", rmd, "\n"))
   rmd_path <- file.path(normalizePath(dirname(rmd)),
      basename(rmd));
   cat(paste0("normalized path: ", rmd_path, "\n"))
   cat(paste0("  file.exists(): ", file.exists(rmd_path), "\n\n"));
}
unlink(rmd_tests)

Output:

 input rmd file: test.rmd
normalized path: /Users/wardjm/temp/test.rmd
  file.exists(): TRUE

 input rmd file: renamed_test.rmd
normalized path: /Users/wardjm/temp/renamed_test.rmd
  file.exists(): TRUE

 input rmd file: testing_rmd/../test.rmd
normalized path: /Users/wardjm/temp/test.rmd
  file.exists(): TRUE

 input rmd file: testing_rmd/renamed.rmd
normalized path: /Users/wardjm/temp/testing_rmd/renamed.rmd
  file.exists(): TRUE

 input rmd file: renamedir_rmd/renamed.rmd
normalized path: /Users/wardjm/temp/testing_rmd/renamed.rmd
  file.exists(): TRUE

This change has the benefit that it respects symlinks that rename the source file, instead of defaulting to use the name of the source file. For example ln -s source.rmd target.rmd will create "target.rmd".

The last test uses a symlink that renames the directory itself. The test still "succeeds" because as far as I can tell, R always resolves a symlinked directory to its absolute path. Also, files will be written to the same place either way, since the directory itself is a symlink.

I don't have a Windows system for testing, I don't think any of this process is relevant to Windows (which I think lacks symlinks - Shortcuts are not the same), although it should work as a drop-in replacement.

This is a naive question: What is a good way to support moving forward? Would I fork, update, test on my own? Then potentially file a pull request? I respect that people have other priorities. :)

[yihui]

The one-liner replacement is shown below:

rmd <- file.path(normalizePath(dirname(rmd)), basename(rmd));

Yes, that's what I'd do.

This is a naive question: What is a good way to support moving forward? Would I fork, update, test on my own? Then potentially file a pull request? I respect that people have other priorities. :)

Sorry for not getting back earlier. Now we have the PR #2438. Please feel free to test it. Thanks!

[jmw86069]

Wow this is great, thanks so much for your time and the updates! I will test as you described in the PR and comment accordingly.

[yihui]

@jmw86069 Have you had a chance to test it? Just asking. No hurry. Thanks!

[jmw86069]

Just to round out this issue, yes @yihui I have finally tested the update, and confirm that it fixes my scenario!

Thank you so much for the update.

I had been using a forked "rogue" version of rmarkdown that I edited for testing, and was quite nervous to submit a PR since it's an important package and pretty core change to its internals. One day I'll issue a PR... probably not this R package, but one day I promise. Haha.

[yihui]

@jmw86069 Totally understand. I was also a little nervous to make this change for the reason you mentioned. The new version of rmarkdown has been on CRAN for two weeks. We haven't received any bug reports yet, and I hope we won't :)

[github-actions[bot]]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.