Pandoc failure during local rendering

[henrif75]

Following the build procedures listed in the README.md page results in pandoc failure:

mdbook serve
2024-03-10 09:56:02 [INFO] (mdbook::book): Book building has started
2024-03-10 09:56:02 [INFO] (mdbook::book): Running the exerciser backend
2024-03-10 09:56:02 [INFO] (mdbook::renderer): Invoking the "exerciser" renderer
2024-03-10 09:56:03 [INFO] (mdbook::book): Running the html backend
2024-03-10 09:56:04 [INFO] (mdbook::book): Running the pandoc backend
2024-03-10 09:56:04 [INFO] (mdbook::renderer): Invoking the "pandoc" renderer
[2024-03-10T16:56:07Z INFO  mdbook_pandoc] Processing redirects in [output.html.redirect]
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: why-rust/an-example-in-c.html => ../hello-world/example.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/why-rust/../hello-world/example.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-2/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-2/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-3/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-3/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: testing/mockall.html => ../android/testing/mockall.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/testing/../android/testing/mockall.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: traits/iterator.html => ../iterators/iterators.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/traits/../iterators/iterators.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-1/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-1/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: basic-syntax/static-and-const.html => ../unsafe-rust/static-and-const.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/basic-syntax/../unsafe-rust/static-and-const.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Unable to normalize link 'comprehensive-rust.pdf' in chapter 'Welcome to Comprehensive Rust 🦀': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/comprehensive-rust.pdf: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../comprehensive-rust-exercises.zip' in chapter 'Compass': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/exercises/bare-metal/../../comprehensive-rust-exercises.zip: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../comprehensive-rust-exercises.zip' in chapter 'RTC Driver': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/exercises/bare-metal/../../comprehensive-rust-exercises.zip: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../borrowing/interior-mutability' in chapter 'Mutex': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/concurrency/shared_state/../../borrowing/interior-mutability: No such file or directory (os error 2)
[WARNING] Could not convert image /private/var/folders/nk/38dlc4z527143grzs9b12cnw0034vl/T/tex2pdf.-6bf89262faae817a/d907026615712cd13ce7e4e5023eb47711d8122f.svg: check that rsvg-convert is in path.
  rsvg-convert: createProcess: posix_spawnp: does not exist (No such file or directory)
[WARNING] Could not convert image /private/var/folders/nk/38dlc4z527143grzs9b12cnw0034vl/T/tex2pdf.-6bf89262faae817a/5180f1968cb395680acc12835345595cdedd9550.svgz: check that rsvg-convert is in path.
  rsvg-convert: createProcess: posix_spawnp: does not exist (No such file or directory)
[WARNING] Could not convert image /private/var/folders/nk/38dlc4z527143grzs9b12cnw0034vl/T/tex2pdf.-6bf89262faae817a/aedd0ef191cf2e1f37d81b92916a869810082d7e.svgz: check that rsvg-convert is in path.
  rsvg-convert: createProcess: posix_spawnp: does not exist (No such file or directory)
lualatex not found. Please select a different --pdf-engine or install lualatex
pandoc exited unsuccessfully
2024-03-10 09:56:10 [ERROR] (mdbook::renderer): Renderer exited with non-zero return code.
2024-03-10 09:56:10 [ERROR] (mdbook::utils): Error: Rendering failed
2024-03-10 09:56:10 [ERROR] (mdbook::utils): 	Caused By: The "pandoc" renderer failed

[djmitche]

It looks like the issue is that rsvg-convert is not installed? I suspect installing that will fix the immediate issue, and perhaps an README update or the like is required to help the next person who encounters this.

I wonder if pandoc just has too many dependencies to expect folks to build locally. Maybe we should be suggesting building the PDF using a docker invocation of some sort?

[henrif75]

I had to install librsvg and lualatex on Mac, but I'm getting another error now:

2024-03-11 14:27:54 [INFO] (mdbook::book): Book building has started
2024-03-11 14:27:55 [INFO] (mdbook::book): Running the exerciser backend
2024-03-11 14:27:55 [INFO] (mdbook::renderer): Invoking the "exerciser" renderer
2024-03-11 14:27:55 [INFO] (mdbook::book): Running the html backend
2024-03-11 14:27:56 [INFO] (mdbook::book): Running the pandoc backend
2024-03-11 14:27:56 [INFO] (mdbook::renderer): Invoking the "pandoc" renderer
[2024-03-11T21:27:57Z INFO  mdbook_pandoc] Processing redirects in [output.html.redirect]
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-1/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-1/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-3/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-3/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: why-rust/an-example-in-c.html => ../hello-world/example.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/why-rust/../hello-world/example.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: testing/mockall.html => ../android/testing/mockall.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/testing/../android/testing/mockall.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: basic-syntax/static-and-const.html => ../unsafe-rust/static-and-const.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/basic-syntax/../unsafe-rust/static-and-const.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: traits/iterator.html => ../iterators/iterators.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/traits/../iterators/iterators.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-2/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-2/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-11T21:28:00Z WARN  mdbook_pandoc::preprocess] Unable to normalize link 'comprehensive-rust.pdf' in chapter 'Welcome to Comprehensive Rust 🦀': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/comprehensive-rust.pdf: No such file or directory (os error 2)
[2024-03-11T21:28:00Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../comprehensive-rust-exercises.zip' in chapter 'Compass': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/exercises/bare-metal/../../comprehensive-rust-exercises.zip: No such file or directory (os error 2)
[2024-03-11T21:28:00Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../comprehensive-rust-exercises.zip' in chapter 'RTC Driver': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/exercises/bare-metal/../../comprehensive-rust-exercises.zip: No such file or directory (os error 2)
[2024-03-11T21:28:00Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../borrowing/interior-mutability' in chapter 'Mutex': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/concurrency/shared_state/../../borrowing/interior-mutability: No such file or directory (os error 2)
	Error producing PDF.
!  ==> Fatal error occurred, no output PDF file produced!

pandoc exited unsuccessfully
2024-03-11 14:28:27 [ERROR] (mdbook::renderer): Renderer exited with non-zero return code.
2024-03-11 14:28:27 [ERROR] (mdbook::utils): Error: Rendering failed
2024-03-11 14:28:27 [ERROR] (mdbook::utils): 	Caused By: The "pandoc" renderer failed

It worked once, then no more. Could we have some sort of config to turn off the PDF generation?

[mgeisler]

@max-heller, do you have experience with building the course on Mac?

@henrif75, you should be able to turn this off by removing the mdbook-pandoc binary you have installed. I get

2024-03-24 21:04:55 [WARN] (mdbook::renderer): The command `mdbook-pandoc` for backend `pandoc` was not found, but was marked as optional.

on my machine because I haven't installed it here.

[max-heller]

@max-heller, do you have experience with building the course on Mac?

Yes, I primarily develop on a Mac.

@henrif75 could you try enabling verbose logging for Pandoc in book.toml and see what it spits out?

[output.pandoc.profile.pdf]
output-file = "comprehensive-rust.pdf"
pdf-engine = "lualatex"
verbosity = "INFO" # ADD THIS LINE

[henrif75]

Here is the output: mdbook.log.gz

[max-heller]

Here is the output: mdbook.log.gz

Do you have Noto fonts installed? The end of the log suggests missing fonts might be the fatal error:

  luaotfload | db : Reload initiated (formats: otf,ttf,ttc); reason: Font "NotoSansMonoCJKSC" not found.
  luaotfload | resolve : sequence of 3 lookups yielded nothing appropriate....texmf-dist/tex/luatex/luaotfload/luaotfload-fallback.lua:50: attempt to inde
  x a nil value (local 'f').
  <to be read again> 
  relax 
  l.154 \begin{document}
                      
   515 words of node memory still in use:
     5 hlist, 1 vlist, 2 rule, 1 dir, 4 kern, 1 glyph, 7 attribute, 60 glue_spec,
   7 attribute_list, 3 if_stack, 1 write, 1 user_defined, 1 pdf_colorstack nodes
     avail lists: 1:1,2:10,3:3,4:1,5:7,7:2,9:4
  !  ==> Fatal error occurred, no output PDF file produced!
  Transcript written on input.log.
Error producing PDF.
!  ==> Fatal error occurred, no output PDF file produced!

pandoc exited unsuccessfully
2024-04-07 17:51:17 [ERROR] (mdbook::renderer): Renderer exited with non-zero return code.
2024-04-07 17:51:17 [ERROR] (mdbook::utils): Error: Rendering failed
2024-04-07 17:51:17 [ERROR] (mdbook::utils): 	Caused By: The "pandoc" renderer failed

I have the following installed through homebrew, although I'm not sure they're all needed:

$ brew list --casks | grep noto
font-noto-color-emoji
font-noto-emoji
font-noto-sans
font-noto-sans-cjk
font-noto-sans-math
font-noto-sans-symbols
font-noto-serif
font-noto-serif-cjk
font-noto-serif-cjk-tc

[henrif75]

It worked when I installed the fonts, but I couldn't find a casket so I manually installed them. Would you have the brew command to fetch these fonts? I'll update the documentation.

[max-heller]

It worked when I installed the fonts, but I couldn't find a casket so I manually installed them. Would you have the brew command to fetch these fonts? I'll update the documentation.

brew tap homebrew/cask-fonts (see here)

[kashopi]

In my Silicon laptop I fixed it by using "tectonic" as a PDF renderer (changing it in the book.toml file) and installing the Noto fonts

[max-heller]

In my Silicon laptop I fixed it by using "tectonic" as a PDF renderer (changing it in the book.toml file) and installing the Noto fonts

tectonic simplifies a lot of dependency wrangling, but If I'm remembering correctly it doesn't support all of the font shenanigans necessary to render multilingual books, so unfortunately comprehensive rust probably can't switch over to it.

https://github.com/google/mdbook-i18n-helpers/issues/200 is trying to figure out a way to skip running optional = true renderers when they aren't needed, which will prevent failures like these

[letesdev]

Hi, I hope you can help me 😄

I am trying to build the book in order to contribute with translating to spanish, but I am facing some errors... I am using a Windows 10 machine and I followed your README.md, CONTRIBUTING.md and TRANSLATIONS.md guides, but still face some errors.

Here are all my steps :

1. I installed Rust by following the instructions on https://rustup.rs/.
2. I cloned this repository
3. I installed the necessary tools (mdbook, mdbook-svgbob, mdbook-i18n-helpers, mdbook-exerciser, and mdbook-course) using cargo.
4. I installed pandoc from Pandoc
5. I installed mdbook-pandoc using cargo, which is not mentioned in the README.md guide.
6. I installed the Noto Fonts from Google Fonts.
7. I installed MikTex to get lualatex from MiKTeX.

Now I am able to build the english version of book with mdbook build, but I can't build the spanish version with MDBOOK_BOOK__LANGUAGE=es mdbook build -d book/es. Here's the output: mdbook_build_es_output1.txt

pandoc: C:\Users\s023793\letesdev\comprehensive-rust\book\es\pandoc\pdf\comprehensive-rust.pdf: withBinaryFile: permission denied (Permission denied)
pandoc exited unsuccessfully
2024-06-04 14:50:33 [ERROR] (mdbook::renderer): Renderer exited with non-zero return code.
2024-06-04 14:50:33 [ERROR] (mdbook::utils): Error: Rendering failed
2024-06-04 14:50:33 [ERROR] (mdbook::utils):    Caused By: The "pandoc" renderer failed

[max-heller]

5. I installed mdbook-pandoc using cargo, which is not mentioned in the README.md guide.

mdbook-pandoc is marked as optional, so if you uninstall it you should be able to build the book without it. https://github.com/max-heller/mdbook-pandoc/pull/93 will soon provide a way to disable mdbook-pandoc outside of CI to avoid having to install its dependencies locally.

pandoc: C:\Users\s023793\letesdev\comprehensive-rust\book\es\pandoc\pdf\comprehensive-rust.pdf: withBinaryFile: permission denied (Permission denied)
pandoc exited unsuccessfully

This is strange, are there restrictive permissions on any of the parent directories of this path?

[letesdev]

Hi max-heller, thanks for the answer 😄

• Are you sure about mdbook-pandoc being optional? Maybe it is possible to build the book without it once max-heller/mdbook-pandoc#93 is implemented, but here's an output of mdbook build before installing it (mdbook_build_output.txt):

  2024-06-04 11:35:24 [WARN] (mdbook::renderer): The command `mdbook-pandoc` for backend `pandoc` was not found, but was marked as optional.
  

• Regarding the issue while building the spanish version: no, there aren't any restrictive permissions on any of the parent directories... you can actually tell by the build of the english version, right ? If I am able to build the english version, shouldn't I be able to build the spanish one ?

[max-heller]

2024-06-04 11:35:24 [WARN] (mdbook::renderer): The command `mdbook-pandoc` for backend `pandoc` was not found, but was marked as optional.

This is a warning, not an error, and shouldn't impact the rest of the build.

If I am able to build the english version, shouldn't I be able to build the spanish one ?

You should be able to, yes. I'll look into it some more

[letesdev]

This is a warning, not an error, and shouldn't impact the rest of the build.

Well, as you can see in the output file (having stderr and stout outputs), the build stops there without any other message... Let me know if you need more details 😄

If I am able to build the english version, shouldn't I be able to build the spanish one ?

You should be able to, yes. I'll look into it some more

Great, thank you so much! 😄

[max-heller]

Well, as you can see in the output file (having stderr and stout outputs), the build stops there without any other message... Let me know if you need more details 😄

At that point, the build has completed successfully. mdbook-pandoc is the last renderer that gets run, so if it's skipped, nothing else is expected. You can verify this by e.g. commenting all of the pandoc configuration in book.toml.

[max-heller]

If I am able to build the english version, shouldn't I be able to build the spanish one ?

You should be able to, yes. I'll look into it some more

Great, thank you so much! 😄

Seems like the error can occur for a variety of reasons (see e.g. here). Do you happen to have the Spanish PDF open while building? An antivirus that might be blocking Pandoc? A network drive/OneDrive setup? Have you tried wiping out the book directory and trying again?

[letesdev]

I am finally able to build the english and the spanish book! Thank you so much @max-heller ! I tried commenting all of the pandoc configuration in the book.toml file as you mentioned, and, just in case, I wiped out the book/ directory and closed OneDrive...

However, I found many of the paragraphs in english in the spanish version are already translated... for instance, after formatting the po/es.po file with dprint fmt po/es.po command, I am able to modify the paragraph in the index.html (I added several 's' to 'Basándonos' just for showing) but not the list :

Do you have a clue in this, by the way ?

[max-heller]

I am finally able to build the english and the spanish book! Thank you so much @max-heller ! I tried commenting all of the pandoc configuration in the book.toml file as you mentioned, and, just in case, I wiped out the book/ directory and closed OneDrive...

Happy to hear it! Just curious, if you reenable the pandoc configuration in book.toml, do you get the same error as before?

However, I found many of the paragraphs in english in the spanish version are already translated... for instance, after formatting the po/es.po file with dprint fmt po/es.po command, I am able to modify the paragraph in the index.html (I added several 's' to 'Basándonos' just for showing) but not the list : Do you have a clue in this, by the way ?

I'm not familiar with the translation infrastructure, this seems like a good question for a separate issue to keep this one on topic

[letesdev]

Just curious, if you reenable the pandoc configuration in book.toml, do you get the same error as before? I tried several builds:

• Wiping out the book/ directory again, closing OneDrive and running mdbook build with the pandoc configuration enabled, the build failed... here's the output: mdbook_build_stderr_stdout_14-06-24_11-07.txt
• Coming from previous run, without wiping, OneDrive closed, the build failed again: mdbook_build_stderr_stdout_14-06-24_11-10.txt
• Coming from previous run, without wiping, OneDrive clsoed and without internet connection, the build seems to have suceded: mdbook_build_stderr_stdout_nowifi_14-06-24_11-11.txt. I didn't see anything wrong in the pdf but here it is in case you want to check something specific: comprehensive-rust.pdf

[max-heller]

• Wiping out the book/ directory again, closing OneDrive and running mdbook build with the pandoc configuration enabled, the build failed... here's the output: mdbook_build_stderr_stdout_14-06-24_11-07.txt
• Coming from previous run, without wiping, OneDrive closed, the build failed again: mdbook_build_stderr_stdout_14-06-24_11-10.txt
• Coming from previous run, without wiping, OneDrive clsoed and without internet connection, the build seems to have suceded: mdbook_build_stderr_stdout_nowifi_14-06-24_11-11.txt. I didn't see anything wrong in the pdf but here it is in case you want to check something specific: comprehensive-rust.pdf

Thanks for testing! This is a specific issue that should be easy to fix, I've opened https://github.com/max-heller/mdbook-pandoc/issues/96 to track it

[mgeisler]

I think we can close this now": We no longer run mdbook-pandoc by default and so people should not see local errors unless they opt-in.