PDF export: 'include::' -> Unresolved directive in <stdin> (path trouble)

[SjoerdV]

trafficstars

I tried to include a file using the relative path (seen from the workspace dir).

include::MyCorp_-_Office_Automation_Notebook/Support/Procedures_-_50_shared_glossary.adoc[leveloffset=1]

Works fine in the VSCode preview.

But delivers this message inside the exported PDF:

Unresolved directive in <stdin> - include::MyCorp_-
_Office_Automation_Notebook/Support/Procedures_-
_50_shared_glossary.adoc[leveloffset=1]

Note there are -_ constructions in the file/path name and these seem to be parsed incorrectly, in the error a newline is generated... I can also not escape the path with ++<filepath>++ this will break both the preview and the pdf. I also tried to make the path an appendix variable like so:

:appendix-a: MyCorp_-_Office_Automation_Notebook/Support/Procedures_-_50_shared_glossary.adoc
:appendix-b: ++MyCorp_-_Office_Automation_Notebook/Support/Procedures_-_50_shared_glossary.adoc++

include::{appendix-a}[leveloffset=1]
include::{appendix-b}[leveloffset=1]

Only appendix-a showed correctly in the Preview. Neither worked in the pdf export (same error as above)

Maybe there is some other path escaping possible here?

VSCode (Linux): 1.74.3 AsciiDoc: v2.9.8

[ggrossetie]

Could you please share a screenshot or the PDF produced? I believe you are using asciidoctor-pdf command-line to generate the PDF? You might need to set the base_dir https://docs.asciidoctor.org/asciidoctor/latest/cli/man1/asciidoctor/#security-settings

[SjoerdV]

Asciidoctor PDF 2.3.4 using Asciidoctor 2.0.18 [https://asciidoctor.org]

@ggrossetie thanks for your suggestion, I added the base_dir in vscode settings, but same error appears. Also tried it as an inline setting in the document. this is the resulting PDF (in the error message it is missing an underscore just after the word 'Procedures' AND just before the first line break (or maybe just the error is being parsed instead of the real path, I don't know)

these are the pdf related vscode settings:

[SjoerdV]

I just found out that when I change the file reference to include parent path statements back to the workspace root folder:

include::../../MyCorp_-_Office_Automation_Notebook/Support/Procedures_-_50_shared_glossary.adoc[leveloffset=1]

The PDF is now rendered correctly, except now the vscode preview does not show the include.

Option: I am perfectly fine adding the parent path references like above, so maybe it can be supported by the js preview? Alternative Option: adding :base_dir: ../../ to the document does nothing at the moment, but if it could be used for the pdf export it would also be fine.

What would you suggest @ggrossetie ?

[SjoerdV]

if I use a snippet as such I can have best of both worlds (preview will render correctly AND pdf export works)... but it might be a little too much overhead ;-)

:external-snippet: MyCorp_-_Office_Automation_Notebook/Support/Procedures_-_50_shared_glossary.adoc
ifdef::env-vscode[]
include::{external-snippet}[leveloffset=1] 
endif::[]
ifdef::backend-pdf[]
include::../../{external-snippet}[leveloffset=1]
endif::[]

[ggrossetie]

So it's a base directory issue, could you please try with the preversion 3.0.3? I made some changes to how asciidoctor-pdf is called.

[SjoerdV]

So it's a base directory issue,

Yes it seems so, I was distracted by the 'formatted' error message printed in the document.

So I think you are referring to this commit: https://github.com/asciidoctor/asciidoctor-vscode/commit/ec3f3ba317eebfceb1bb80b6b87214f6ee4bc8ba (from July 2022)

OK I'll do some beta testing for ya ;-) The aim was to have this statement work in both the preview and the pdf export alike:

include::MyCorp_-_Office_Automation_Notebook/Support/Procedures_-_50_shared_glossary.adoc[leveloffset=1]

I tried it out but didn't work as expected:

• I still get the exact same error message in the pdf (preview works correctly)
• the settings asciidoc.preview.attributes seem to be replaced with asciidoc.preview.asciidoctorAttributes, but styles do not load anymore in the preview
• preview now does automatic TOC loading, without :toc: in main document (otherwise a double TOC appears in preview)
• pdfexport does not show TOC because of missing :toc: in main document

Maybe there is some manual or guide for upgrading to 3+ release, I missed?

these are the new workspace settings I would use in the preview

    ...
    "asciidoc.preview.useEditorStyle": false,
    "asciidoc.useWorkspaceRootAsBaseDirectory": true,
    "asciidoc.preview.asciidoctorAttributes": {
      "styles": "",
      "imagesdir": "resources/media",
      "stylesdir": "resources/css",
      "stylesheet": "my-asciidoc-styles.css"
    },
    "asciidoc.preview.scrollEditorWithPreview": true,
    "asciidoc.preview.scrollPreviewWithEditor": true

these are the old workspace settings I would use in 2.9.8

    ...
    "asciidoc.trace": "verbose",
    "asciidoc.use_asciidoctorpdf": true,
    "asciidoc.useWorkspaceRoot": true,
    "asciidoc.preview.attributes": {
      "styles": "",
      "imagesdir": "resources/media",
      "stylesdir": "resources/css",
      "stylesheet": "my-asciidoc-styles.css"
    },
    "asciidoc.use_asciidoctor_js": true,
    "asciidoc.preview.useEditorStyle": false

Just a thought: wouldn't it be a lot easier if the --base-dir argument for asciidoctor-pdf could be (automatically) set same as the VSCode workspace directory. In the settings dialog it is stated that By default, it passes the following arguments: --quiet and --base-dir with the full directory path to the AsciiDoc document. Could that be the issue?

[ggrossetie]

Can you please share your directory structure? Also base_dir can only be overriden from the CLI (or API) so you will need to configure --base-dir=/path/to/my/basedir .

If the include directive is used in the primary (top-level) document, relative paths are resolved relative to the base directory. (The base directory defaults to the directory of the primary document and can be overridden from the CLI or API).

Does MyCorp_-_Office_Automation_Notebook/Support/Procedures_-_50_shared_glossary.adoc is in the same directory as your primary document?

Just a thought: wouldn't it be a lot easier if the --base-dir argument for asciidoctor-pdf could be (automatically) set same as the VSCode workspace directory.

It depends. Some users will prefer to resolve relative paths from the directory of the primary document and others from the VSCode workspace directory. We might need to add an option in the settings.

[SjoerdV]

So this is my tree.

├── MyCorp_-_Office_Automation_Notebook
│   ├── [More Dirs]
│   ├── Features
│   │   └── Features_-_50_shared_summary.adoc
│   └── Support
│   │   ├── Procedures_-_01_Introduction.adoc
│   │   └── Procedures_-_50_shared_glossary.adoc
├── resources
│   ├── css
│   ├── fonts
│   ├── media
│   │   └── icons
│   │── themes
│   └── default_settings.adoc
└── vscode_notebook
    └── pyaes

all documents in the 'MyCorp_-_Office_Automation_Notebook' structure will always include the resources/default_settings.adoc. These contain all the generic settings.

Additionally, any document in the MyCorp_-_Office_Automation_Notebook structure may include appendices/addenda (usually from the same folder the main document is in., but these may come from other folders as well)

In this case the document Procedures_-_01_Introduction.adoc includes default_settings.adoc,Procedures_-_50_shared_glossary.adoc and Features_-_50_shared_summary.adoc

[ggrossetie]

I think we made a mistake in the VS code extension since includes should be resolved relative to the current document.

In this case the document Procedures_-01_Introduction.adoc includes default_settings.adoc,Procedures-50_shared_glossary.adoc and Features-_50_shared_summary.adoc

The correct syntax is:

Procedures_-_01_Introduction.adoc

include::../resources/default_settings.adoc[]

include::Procedures_-_50_shared_glossary.adoc[]

include::../Features/Features_-_50_shared_summary.adoc[]

Otherwise, we will get a different result when using the CLI. I asked @mojavelinux about this issue on https://asciidoctor.zulipchat.com/#narrow/stream/279642-users/topic/Base.20directory.20and.20IDE (please feel free to add anything I may have missed in this thread)

[SjoerdV]

Beforehand: In your examples there should be one more level of 'parent' to match my example tree, so:

include::../../resources/default_settings.adoc[]

include::Procedures_-_50_shared_glossary.adoc[]

include::../../Features/Features_-_50_shared_summary.adoc[]

Things I learned and recommend so far:

1. I have reverted back to the release version, because the preview had too many issues (with toc and preview styles not loading period, whatever I did...) -> what follows is based on the current release 2.9.8 version
2. Taking a step back and looking at all the pathing issues (themes, styles, images, includes,etc.), what seems to be making this very confusing is the actual setting that was created to alleviate 'pathing' issues -> asciidoc.useWorkspaceRoot (in the preview it is renamed, but still present). This setting seems to only works for the 'JS' parts of the entire workflow and not for 'ascidoctor-pdf' part. So keeping this setting as false to make the whole workflow work (without elaborate constructions) is what I'll be doing next.
3. Also to make things more uniform and more 'ready for change' INCLUDES in the same dir should be managed the same as in other part of the tree. Treating those the same makes it easier in the long run to maintain IMHO
4. Some parts of the workflow (underlying systems) only behave well when absolute paths or relative paths (with parent path backtracking) are used, so we'll just have to use them IMHO
5. the :parent-path: attribute declaration in the main document and its use {parent-path} everywhere else is what makes this all work
6. What I didn't test or make use of are nested includes, so that might be something to look into at some point, maybe just overriding the :parent-path: attribute is sufficient... There are a lot of rabbit holes to follow ;-)

EXAMPLE VSCODE SETTINGS

    "asciidoc.trace": "verbose",
    "asciidoc.use_asciidoctorpdf": true,
    "asciidoc.useWorkspaceRoot": false,
    "asciidoc.preview.attributes": {
      "styles": "",
      "imagesdir": "${workspaceFolder}/resources/media",
      "stylesdir": "${workspaceFolder}/resources/css", // used for preview and in pdfexport when 'use_asciidoctorpdf = false'
      "stylesheet": "my-asciidoc-styles.css" // used for preview and in pdfexport when 'use_asciidoctorpdf = false'
    },
    //"asciidoc.preview.style": "material-green.css",
    "asciidoc.use_asciidoctor_js": true,
    "asciidoc.preview.useEditorStyle": false,

MAIN DOCUMENT Procedures_-_01_Introduction.adoc

= Procedures - Introduction
// Set Document Properties
:doctitle: Procedures - Introduction
:author: My Name
// Include Generic Settings
:parent-path: ../../
include::{parent-path}resources/default_settings.adoc[]

toc::[]

include::{parent-path}MyCorp_-_Office_Automation_Notebook/Support/Procedures_-_50_shared_glossary.adoc[leveloffset=1]

include::{parent-path}MyCorp_-_Office_Automation_Notebook/Features/Features_-_50_shared_summary.adoc[leveloffset=1]
...

INCLUDE settings in workspaceroot default_settings.adoc

// Path Settings  (styles* settings are only used for HTML export)
:imagesdir: {parent-path}resources/media
:stylesdir: {parent-path}resources/css
:stylesheet: my-asciidoc-styles.css
// Set Theme Configuration (used for PDF export)
:pdf-themesdir: {parent-path}resources/themes
:pdf-theme: my
:pdf-fontsdir: {parent-path}resources/fonts
// Use Local Font (Awesome, Web) fonts
:icons: font
// Store data locally on Export
:data-uri:
:allow-uri-read:
// Set Specific Variables
:showtitle:
// Set Table of Contents config
:toc: macro
:toclevels: 5
:toc-title: Contents
...

INCLUDE in same dir Procedures_-_50_shared_glossary.adoc

[#_glossary]
= Glossary

We explain the following *terms* which are used throughout this document:
...

INCLUDE in other dir Features_-_50_shared_summary.adoc

[#_features]
= Features

We explain the following *features* which are referenced in this document:
...

Concluding. To make the experience better it would be nice the parent-path attribute would just be calculated as seen from a base-dir (ie. workspace dir) for all aspects in this workflow (both js and asciidoctor-pdf), as you just need to calculate the relative 'depth' of a path and adding ../for every level. That way I don't have to bother with adding the parent-path attribute myself in every document and calculate the relative depth everytime. Just a thought though.

[ggrossetie]

I have reverted back to the release version, because the preview had too many issues (with toc and preview styles not loading period, whatever I did...) -> what follows is based on the current release 2.9.8 version

Could you please create issues (if they don't exist already) on what's not working on 3.x as we will "soon" release a "final" 3.x version?

Concluding. To make the experience better it would be nice the parent-path attribute would just be calculated as seen from a base-dir (ie. workspace dir) for all aspects in this workflow (both js and asciidoctor-pdf), as you just need to calculate the relative 'depth' of a path and adding ../for every level

The VS Code extension could compute an attribute but then it won't work on the CLI. The technique you are using is (for now) the recommended approach:

If you store your AsciiDoc files in nested folders at different levels, relative file paths can quickly become awkward and inflexible. A common pattern to help here is to define the paths in attributes defined in the header, then prefix all include paths with a reference to one of these attributes:

:includedir: _includes
:sourcedir: ../src/main/java

include::{includedir}/fragment1.adoc[]

[source,java]
----
include::{sourcedir}/org/asciidoctor/Asciidoctor.java[]
----

https://docs.asciidoctor.org/asciidoc/latest/directives/include/#include-resolution

What I didn't test or make use of are nested includes, so that might be something to look into at some point, maybe just overriding the :parent-path: attribute is sufficient... There are a lot of rabbit holes to follow ;-)

Please keep in mind that, if the include directive is used in a file that has itself been included, the path is resolved relative to the including (i.e., current) file.

[SjoerdV]

Great! Eventually it's just a matter of RTFM. Arghh. Nested includes should just work with that strategy, so thanks for that.

Would be nice if these 'best practices' were a bit more easy to find and not somewhere in the middle of a page. Something like tagging a region with 'recommended-practice' would be nice? and then collecting all these best-practices and put in one document like here: https://asciidoctor.org/docs/asciidoc-recommended-practices

The VS Code extension could compute an attribute but then it won't work on the CLI.

Wait... let's just say that I would set useWorkspaceRoot=true . Wouldn't it be possible for the extension to update an attribute of my choice (should be an extra setting), if exists, let's say :parent-path: in the active document, on save? That way it will also work on the CLI.

[ggrossetie]

Great! Eventually it's just a matter of RTFM. Arghh. Nested includes should just work with that strategy, so thanks for that.

Would be nice if these 'best practices' were a bit more easy to find and not somewhere in the middle of a page. Something like tagging a region with 'recommended-practice' would be nice? and then collecting all these best-practices and put in one document like here: https://asciidoctor.org/docs/asciidoc-recommended-practices

That's a good idea! This page has not yet been migrated to the new documentation site but feel free to discuss this change in the project chat: https://chat.asciidoctor.org and submit a pull request.

Wait... let's just say that I would set useWorkspaceRoot=true...

I still think that setting the base_dir option (i.e. not an attribute) on the command line would work too:

asciidoctor-pdf --base-dir=/path/to/workspace file.adoc

I need to debug a little more on the 3.x version to understand why it's not working.