adrienbrignon
Timeout Error When Using Covers Functionality
Dear @adrienbrignon
Thank you for creating this great plugin. At this time I have some issues with using the covers functionality. I saw #36 but with the configuration as in this comment it doesn't work either.
My Configuration
plugins:
- exporter:
logging:
level: debug
formats:
pdf:
enabled: !ENV [MKDOCS_EXPORTER_PDF, true]
concurrency: 20
browser:
timeout: 120000
stylesheets:
- resources/stylesheets/pdf.scss
covers:
front: resources/templates/covers/front.html.j2
back: resources/templates/covers/back.html.j2
aggregator:
enabled: !ENV [MKDOCS_EXPORTER_PDF_AGGREGATOR, true]
output: documentation.pdf
covers: all
Without covers the plugin will work.
INFO - [mkdocs-exporter.pdf] Launching browser...
INFO - [mkdocs-exporter.pdf] Rendering 'second.md'...
INFO - [mkdocs-exporter.pdf] Rendering 'third.md'...
INFO - [mkdocs-exporter.pdf] Aggregating pages to 'documentation.pdf'...
INFO - Documentation built in 23.91 seconds
INFO - [macros] - We will also watch: ['.']
INFO - [11:35:24] Watching paths for changes: 'docs', 'mkdocs.yml', '.', 'resources/stylesheets/pdf.scss'
INFO - [11:35:24] Serving on http://127.0.0.1:8000/
INFO - [11:35:27] Browser connected: http://localhost:8000/second.html
Error (when covers are enabled)
INFO - [mkdocs-exporter.pdf] Rendering 'index.md'...
INFO - [mkdocs-exporter.pdf] Launching browser...
INFO - [mkdocs-exporter.pdf] Rendering 'second.md'...
INFO - [mkdocs-exporter.pdf] Rendering 'third.md'...
ERROR - [mkdocs-exporter.pdf.browser] (error) mkdocs example
Failed to load resource: net::ERR_FILE_NOT_FOUND
ERROR - [mkdocs-exporter.pdf.browser] (error) Third - mkdocs example
Failed to load resource: net::ERR_FILE_NOT_FOUND
ERROR - [mkdocs-exporter.pdf.browser] (error) Second - mkdocs example
Failed to load resource: net::ERR_FILE_NOT_FOUND
ERROR - [mkdocs-exporter.pdf.browser] (error) mkdocs example
Failed to load resource: net::ERR_FILE_NOT_FOUND
ERROR - [mkdocs-exporter.pdf.browser] (error) Second - mkdocs example
Failed to load resource: net::ERR_FILE_NOT_FOUND
Traceback (most recent call last):
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/bin/mkdocs", line 8, in <module>
sys.exit(cli())
^^^^^
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/mkdocs/__main__.py", line 272, in serve_command
serve.serve(**kwargs)
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/mkdocs/commands/serve.py", line 85, in serve
builder(config)
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/mkdocs/commands/serve.py", line 67, in builder
build(config, serve_url=None if is_clean else serve_url, dirty=is_dirty)
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/mkdocs/commands/build.py", line 347, in build
config.plugins.on_post_build(config=config)
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/mkdocs/plugins.py", line 602, in on_post_build
return self.run_event('post_build', config=config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/mkdocs/plugins.py", line 568, in run_event
result = method(**kwargs)
^^^^^^^^^^^^^^^^
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/mkdocs_exporter/formats/pdf/plugin.py", line 154, in _on_post_build_1
self.loop.run_until_complete(asyncio.gather(*concurrently(self.tasks, max(1, self.config.concurrency or 1))))
File "/usr/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/mkdocs_exporter/helpers.py", line 29, in limit
return await asyncio.create_task(coroutine)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/mkdocs_exporter/formats/pdf/plugin.py", line 135, in render
pdf, pages = await self.renderer.render(html)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/mkdocs_exporter/formats/pdf/renderer.py", line 91, in render
return await self.browser.print(html)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/mkdocs_exporter/formats/pdf/browser.py", line 100, in print
await context.locator('body[mkdocs-exporter="true"]').wait_for(timeout=self.timeout)
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/playwright/async_api/_generated.py", line 17738, in wait_for
await self._impl_obj.wait_for(timeout=timeout, state=state)
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/playwright/_impl/_locator.py", line 679, in wait_for
await self._frame.wait_for_selector(
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/playwright/_impl/_frame.py", line 323, in wait_for_selector
await self._channel.send("waitForSelector", locals_to_params(locals()))
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 61, in send
return await self._connection.wrap_api_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dev/mkdocs-materials-with-pdf-export/.venv/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 528, in wrap_api_call
raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
playwright._impl._errors.TimeoutError: Locator.wait_for: Timeout 120000ms exceeded.
Call log:
- waiting for locator("body[mkdocs-exporter=\"true\"]") to be visible
(.venv) dev@work:~/mkdocs-materials-with-pdf-export$
Unfortunately, the timeout argument does not work as expected. I have a documentation with about 100 pages, which also contains a few pictures.
Looks like the mkdocs-exporter=true attribute is set via JavaScript in the plugin. Not sure why covers would break that.
A few things to try:
- Disable timeout by setting
formats.pdf.browser.timeout: 0 - Disable concurrency by setting
formats.pdf.concurrency: 1
These are the settings I use. However, I have a much larger project on a different Python version and weird hardware. Concurrency usually causes exceptions for me and I don't know why, so I am generally suspicious against it when it comes to weird errors.
A few things to try to try finding the culprit:
- Remove everything from the cover files (so they are just empty files)
- Reduce the cover files to pure HTML (no CSS or Jinja)
- Reduce the cover files to just Jinja and HTML
- Reduce the cover files to just CSS and HTML
Are you able to run this repo's mkdocs without issue with the cover pages?
Thanks @nbanyan for your very fast reply.
I will test it as soon as possible.
Thank you very much for your advice @nbanyan
I have done a lot of tests. I have set the concurrency to 1 and the timeout to 0, as you suggested. Nevertheless, the creation of the PDFs is stalling. If I deactivate the back covers, the creation works successfully. If I only have the back covers active, the creation does not work either. In my case it is due to the back covers.
Have you ever noticed this?
EDIT: I have also noticed that the mermaid diagrams are displayed online, but not in the PDF. Do you have any advice?
plugins:
- .... # Some other plugins
- mermaid2
# Markdown Extensions
markdown_extensions:
- pymdownx.highlight:
anchor_linenums: true
- pymdownx.inlinehilite
- pymdownx.snippets
- admonition
- pymdownx.arithmatex:
generic: true
- footnotes
- pymdownx.details
- pymdownx.superfences
- pymdownx.mark
- attr_list
- md_in_html
- pymdownx.blocks.caption
- pymdownx.tabbed:
alternate_style: true
- pymdownx.emoji:
emoji_index: !!python/name:material.extensions.emoji.twemoji
emoji_generator: !!python/name:material.extensions.emoji.to_svg
- pymdownx.superfences:
# make exceptions to highlighting of code:
custom_fences:
- name: mermaid
class: mermaid
format: !!python/name:mermaid2.fence_mermaid_custom
For most things JavaScript, exporter needs to be told to run it specifically. The script for Mermaid and a few others are in the docs/assets/scripts directory.
I haven't experimented with back covers much yet. The ones I use are just empty <div> tags. The example ones in this repo aren't much more complex either. What elements/features do you use in your back cover?
Also, PyMdown Extensions has its own formatting for Mermaid now, so you don't need the older mermaid2 plugin.
markdown_extensions:
- pymdownx.superfences:
preserve_tabs: true
custom_fences:
# Mermaid diagrams
- name: mermaid
class: mermaid
format: !!python/name:pymdownx.superfences.fence_code_format
Thanks for the tip. the mermaid diagrams are now displayed correctly in the pdf .
For test reasons, I used the template from the repository adrienbrignon/mkdocs-exporter. I deleted the version and the background image from front and back.
Like other people, I prefer to use Gitlab (#43) and I have a small problem during the deployment process. The documentation can be opened via user.gitlab.io/doc. If I then want to download the PDF, for example, it opens it under user.gitlab.io/index.pdf. This is wrong. It should actually be user.gitlab.io/doc/index.pdf. I have researched on the internet how to solve the problem, but have not found the right solution.
Can you help me with this?
There is a config for specifying a prefix for the pdf URLs for the buttons' !!python/name:mkdocs_exporter.formats.pdf.buttons.download.href scripting option.
https://github.com/adrienbrignon/mkdocs-exporter/blob/0d4eae0c1920ecffe87d66b7f404c13ff7cdea47/mkdocs_exporter/formats/pdf/buttons/download.py#L15
https://github.com/adrienbrignon/mkdocs-exporter/blob/0d4eae0c1920ecffe87d66b7f404c13ff7cdea47/mkdocs_exporter/formats/pdf/config.py#L71
So something like this may make the gitlab hosted site path better:
plugins:
- exporter:
formats:
pdf:
url: doc
Thank you for your message. I have tested this.
My config:
- exporter:
logging:
level: debug
formats:
pdf:
enabled: !ENV [MKDOCS_EXPORTER_PDF, true]
url: m123_nh
concurrency: 1
browser:
timeout: 0
stylesheets:
- resources/stylesheets/pdf.scss
covers:
front: resources/templates/covers/front.html.j2
aggregator:
enabled: !ENV [MKDOCS_EXPORTER_PDF_AGGREGATOR, true]
output: Lernjournal_m123_Niclas-Heinz.pdf
covers: front
buttons:
- title: View as PDF
icon: material-file-move-outline
enabled: !!python/name:mkdocs_exporter.formats.pdf.buttons.download.enabled
attributes:
target: _blank
href: !!python/name:mkdocs_exporter.formats.pdf.buttons.download.href
- title: Download as PDF
icon: material-file-download-outline
enabled: !!python/name:mkdocs_exporter.formats.pdf.buttons.download.enabled
attributes: !!python/name:mkdocs_exporter.formats.pdf.buttons.download.attributes
but it still tries to open the links with /index.pdf instead of /m123_nh/index.pdf
My project structure
docs
├── index.md
└── test
└── test.md
However, when I go to test/index.html and then download the pdf, it shows me /m123_nh/test.pdf instead of m123_nh/test/test.
Do you have any idea what this could be due to?
Apparently the formats.pdf.url used in download.py is not the value from the config but rather the path for that page's PDF relative to docs_dir. I haven't been able to find where that config option is actually used.
The href method itself is wrong since both arguments to relpath are files, but relpath assumes the second argument is a directory, so we end up with an extra ../ prepended to the URL.
Theoretically you could create a Python file with a custom href method to use instead.
plugins:
- exporter:
buttons:
- title: Download as PDF
icon: material-file-download-outline
attributes:
href: !!python/name:my_module.href
my_module.py:
import os
from mkdocs_exporter.page import Page
def href(page: Page, **kwargs) -> str:
"""The value of the 'href' attribute."""
# Custom website sub-directory to prepend to URLs
url_prefix = 'm123_nh'
# Add relative pathing to docs_dir
for i in range(len(page.ancestors)):
url_prefix = os.path.join(os.path.pardir, url_prefix)
# Append the PDF URL for the page and normalize
return os.path.normpath(os.path.join(url_prefix, page.formats['pdf']['url']))
I get 'module not found' errors when I try this though, so the alternative is to edit the download.py file directly in your Python environment (or better from the mkdocs-exporter directory installed via pip install -e .
@nbanyan Many thanks for the support and for creating the issues
I have found a project hosted on gitlab
@AdrianDC how did you solve the problem?
I use mkdocs, mkdocs-material and mkdocs-exporter for almost all my public (RadianDevCore) projects and some professional documentations (work).
You can look any of these, the configurations are almost identical : https://gitlab.com/RadianDevCore/tools
Feel free to check my GitLab CI configurations, containers sources, mkdocs configurations and docs sources.
@AdrianDC
I have looked at your configuration. It's actually the same as mine. Except for one plugin: ‘offline’. After I deactivated this, everything worked smoothly.
@AdrianDC @nbanyan Thank you for your help and have a great festive season
That offline causes the issue is interesting. The plugin's page says it conflicts with other things that use the Fetch API (https://squidfunk.github.io/mkdocs-material/plugins/offline/#limitations). Playwright uses fetch, yet I am able to enable the offline plugin without incurring any errors.
Does your project (and specifically the back cover) cause any references to external resources? This might be as hidden as Google's fonts-gstatic-com or fontawesome (both of which I run from a local folder).
Sorry I have forgot to answer :(
No, my backcovers do not have any external resources. If I only activate front (and disable offline), the creation works without any problems.
I'm having this same issue, but it doesn't appear to be cover-related.
My config:
- exporter:
logging:
level: debug
formats:
pdf:
enabled: !ENV [MKDOCS_EXPORTER_PDF, true]
concurrency: 1
explicit: true
covers:
front: export/covers/front.html
back: export/covers/back.html
aggregator:
enabled: true
output: test.pdf
covers: none
I set explicit to true, and have only one page index.md set to export. The cover files are empty html files. The result is always:
Call log:
- waiting for locator("body[mkdocs-exporter=\"true\"]") to be visible
If I set the timeout to 0 as suggested above, the build just hangs forever.
I've tried disabling all other plugins, theme features, and markdown extensions.
I also tried running headless: false and I get the following javascript errors in the browser immediately on launch, though they may be red-herrings:
index.ts:77 Uncaught ReferenceError: Missing element: expected "[data-md-component=header]" to be present
at R (index.ts:77:11)
at Se (index.ts:121:10)
at bundle.ts:209:29
at bundle.ts:318:21
tmp9qk2um3j.html:1042 Uncaught (in promise) SyntaxError: Failed to execute 'querySelectorAll' on 'DocumentFragment': ' h5)' is not a valid selector.
at e.processSelectors (tmp9qk2um3j.html:1042:41759)
at e.afterParsed (tmp9qk2um3j.html:1042:41678)
at tmp9qk2um3j.html:265:4246
at Array.forEach (<anonymous>)
at eC.trigger (tmp9qk2um3j.html:265:4218)
at tn.flow (tmp9qk2um3j.html:337:2743)
at async ps.preview (tmp9qk2um3j.html:1042:67888)
at async tmp9qk2um3j.html:1042:68709
Running on an M2 mac (15.5 Sequoia) MkDocs version 1.6.1 (MkDocs Material) Python v 3.9.6
