Support Antora

[Skeeve]

Yesterday, 2021-08-22 I learned about Antora at FrOSCon 2021. It was presented by the developer of the IntelliJ Asciidoctor Plugin.

While I'm not yet using Asciidoctor, I only learned about it a day ago ;), It sounds quite promising and useful, what he presented. Unfortunately there is no recording available yet.

Features

workspace

• [x] #695

antora.yml

• [x] detect antora.yml files in the workspace and ask user if they want to enable Antora support

site-manifest.json

• [ ] allow to configure a path to a site-manifest.json file
• [ ] read and parse the site-manifest.json file to provide autocompletion on pages (xref, include, link...)

Preview

• [x] #691
• [x] resolve image resource IDs in the preview
• [x] resolve include resource IDs in the preview
• [ ] resolve xref resource IDs in the preview

Navigate

• [ ] #692
• [ ] support "Go to" (cmd+click) on xref resource IDs (for instance, xref:module:filename.adoc[])
• [ ] support "Go to" (cmd+click) on include resource IDs (for instance, include::include::ROOT:partial$treeline-warning.adoc[])

Auto-completion

• [ ] #693
  • images within the same module
  • images from other modules
  • images from other components
• [ ] add includes auto-completion in this order
  • attachments within the same module
  • attachments from other modules
  • attachments from other components
• [ ] add xref auto-completion in this order
  • pages within the same module
  • pages from other modules
  • pages from other components

Suggestions/actions

• [ ] transform a resource ID to a fully qualified ID (for instance, from image:seaswell.png[] to image:2.0@cli:commands:seaswell.png[])
• [ ] suggest to simplify a resource ID (for instance, from image:2.0@cli:commands:seaswell.png to image:seaswell.png[] if the image is referenced in the same component/module/version)

[aisbergde]

Antora support in the IntelliJ Asciidoctor Plugin is excellent and was the only reason for me to install and use IntelliJ.

But my main editor is VSC and of course I would like to do also much more in VSC when working with asciidoc files.

Another point is: It is very easy to install VSC also without admin rights and the asciidoctor-vscode plugin even works in Azure Data Studio. And it is always allowed to install Azure Data Studio in my projects, because it is installed together with SSMS (SQL Server Management Studio). So normally I find ways to install and use this VSC Plugin, but it is much harder to be allowed to install and use IntelliJ.

[Skeeve]

The recording: https://media.ccc.de/v/froscon2021-2656-online-dokumentation_fur_nutzer_mit_asciidoc_und_antora

German only :(

[danyill]

It would be good to have a list of capabilities that mean "supporting Antora" so that we can progress them. I'm beginning to use Antora on a project and would like some of these capabilities also.

We have quite a lot of issues requesting support now (#230, #358, #380 and #409). Perhaps we can use this issue to scope out some requirements.

What are the most important things? From the current issues it seems to be:

• Processing a resource id in images, pages (xrefs and links)
• Being able to link to a production site?
• Do we need to be able to reference a playbook file and have vs-code open other folders/workspaces for different components?

How do achieve this concretely?

• Do we re-implement Antora functions (resource id processing might be something we can just "pull in"?) or do we effectively bundle Antora into this extension?
• Do we use our own UI or wait until Antora 3 and use extensions to gain the content and perhaps reduce the level of processing used (we probably don't need to build a full UI -- or do we ).

I don't speak German... I also haven't used the IntelliJ tool -- however there is a good feature list on the extension docs which is probably where we should start:

https://intellij-asciidoc-plugin.ahus1.de/docs/users-guide/features/advanced/antora.html

[anb0s]

I'm investigating the AsciiDoc with Antora as alternative solution to Markdown with MkDocs for middle and big architecture documentation projects (many products, components, branches etc.). I'm also investigating the issues with relative image rendering (PlantUML, Vega etc.), completion support for xrefs and others in VSCode.

@Skeeve the video above now encouraged me to continue investigating the issues and help with reporting, testing and may be resolving them...

@danyill in the Q&A session from the video the maintainer (Alexander Schwartz) of IntelliJ AsciiDoc plugin is talking about sharing knowledge and support (chat, online meeting etc.) if similar features are needed in VSCode and somebody wants to implement them :)

Lets start with the feature list and known issues...

[ggrossetie]

It would be good to have a list of capabilities that mean "supporting Antora" so that we can progress them. I'm beginning to use Antora on a project and would like some of these capabilities also.

💯 That would be extremely useful! Maybe @ahus1 can help us since the IntelliJ plugin has an excellent support for Antora.

I've created a similar issue on the browser extension: https://github.com/asciidoctor/asciidoctor-browser-extension/issues/299.

I guess the first would be to detect that we are in an Antora module (probably locating the antora.yml file). If that's the case, then we should use a specific logic to resolve external resources (images, partials, examples...).

[ahus1]

Hi @Mogztter - thanks for summoning me.

I am happy to support in answering questions like "How did you..." and "Why did you...", also "What did you do before/after...". Pick my brain :-)

Regarding the video: I pre-recorded the same talk in English for DevConf.US and also FOSDEM, see https://www.youtube.com/watch?v=MG-sajf2PP4 for details. As it was pre-recorded, there is no Q&A at the end. DevConf.US published a recording that included the Q&A as well, see https://www.youtube.com/watch?v=LT0a--DNJhI (but the quality of the pre-recorded part is not as good there).

Regarding features: I learned when implementing it for the IntelliJ plugin that the functionality can be grown incrementally. Users will provide feedback on early functionality. Ideally there is fast feedback (think earth-sky as described by the ZertoMQ community). For the IntelliJ plugin, @danhaywood picked "sky", shared his needs, did a lot of tests and provided valuable input that steered development.

Adding to @Mogztter's thoughts: recognizing that an AsciiDoc file is part of a component by locating an antora.yml file is definitely the first step. What could be next is to set some attributes for the preview like :icons: font (for nicer admonitions) as Antora does this by default, and setting the imagesdir to the module's folder so that images show up in the preview.

Functionality could then be added bit by bit: Picking up attributes from the component descriptor, settings more Antora attributes, picking up page names for xrefs, etc.

For the IntelliJ plugin I decided to imitate Antora's behavior by re-implementing some of the behavior, for example resolving includes and xrefs. I can show where I hooked into Asciidoctor to inject the necessary functionality.

When growing functionality, some features targeted the editor for auto-completion and inspections, others targeted the preview. For a start, it was supporting only references within the same component. Only later I added functionality to resolve cross-component references.

This week and next I'll have only time to comment on issues. From Sep 27 onwards I'm happy to join a synchronous communication (video call, etc.)

[LukeCz]

Hi, it would be great to include 2 features from Antora to this plugin: extended syntax for include and image macros, basically ability to specify module.

[MLNW]

Native support for Antora would be a great addition to this plugin!

[apupier]

some Antora support has been provided with this PR https://github.com/asciidoctor/asciidoctor-vscode/pull/580 it is not part of a released version (yet). I do not know the features that are covered. @marieflorescontact Could you provide some information on it?

[marieflorescontact]

For now the extension detects the antora.yml file and offers the user to activate the Antora support (#580).

I am currently working on the following features:

• Resoving Antora Ids for the following macros: image, video, xref (include seems a little bit more complicated 🤔)
• Drop into editor using this api to create a macro depending on the file type. (#624)

[aisbergde]

When is a preview version planned? I would like to test this Antora feature.

[omfgnuts]

Any progress has been done on this matter?

Would definitely love to come back to VS Code, Antora support in WebStorm is the only thing holding me on this java monstrosity

[ggrossetie]

Yes, some work has been done in 3.x (pre-release) but we are still missing a lot of features. I think we should create dedicated issues for each feature otherwise it's too daunting.

We can use this issue as a "meta issue"

[eddennison]

I am not seeing image resource ID resolution working in the current plug-in, even in the simplest case where the image is stored in the standard images directory in the current module:

Antora build output:

Is this supposed to be working? References to other modules in the current component or modules in other components also don't seem to work. (I was hoping to use this code as a model for getting include directives working.)

[ggrossetie]

Did you enable Antora support? What version are you using?

[eddennison]

Did you enable Antora support? What version are you using?

3.1.5 with Antora support enabled. I also have allow-uri-read set in asciidoctorAttributes.

[ggrossetie]

Could you please share a repository where I can reproduce this issue? It's working fine on my projects.

[eddennison]

Could you please share a repository where I can reproduce this issue? It's working fine on my projects.

It's a private repo in a corporate environment. Do you have a test repo that you can share? That would let me identify if there were an issue with my system or environment. The demo Antora demo repo has no images.

I need also to do an experiment -- we use LFS for binary data, and that may be the problem.

[ggrossetie]

Are you using Windows? Could you please toggle the Developer Tools and copy/paste what you see in the "Console" tab when you open your AsciiDoc file?

[eddennison]

Are you using Windows? Could you please toggle the Developer Tools and copy/paste what you see in the "Console" tab when you open your AsciiDoc file?

[Extension Host] Unable to find an applicable Antora configuration file in [file:///c%3A/Users/ed/antora/sw_is.ad/isa/antora.yml, file:///c%3A/Users/ed/antora/common.ad/doc/antora.yml, file:///c%3A/Users/ed/antora/hw_pnp.ad/x80/antora.yml, file:///c%3A/Users/ed/antora/common.ad/snippets/antora.yml] for the AsciiDoc document file:///c%3A/Users/ed/antora/common.ad/doc/modules/ROOT/pages/goodbye.adoc
console.ts:137 [Extension Host] Unable to find an applicable Antora configuration file in [file:///c%3A/Users/ed/antora/sw_is.ad/isa/antora.yml, file:///c%3A/Users/ed/antora/hw_pnp.ad/x80/antora.yml, file:///c%3A/Users/ed/antora/common.ad/snippets/antora.yml, file:///c%3A/Users/ed/antora/common.ad/doc/antora.yml] for the AsciiDoc document file:///c%3A/Users/ed/antora/common.ad/doc/modules/ROOT/pages/goodbye.adoc
resourceLoading.ts:82 Error: Unable to read file 'c:\Users\ed\antora\common.ad\doc\modules\ROOT\pages\same-in-english.jpg' (Error: Unable to resolve nonexistent file 'c:\Users\ed\antora\common.ad\doc\modules\ROOT\pages\same-in-english.jpg')
    at r.w (fileService.ts:607:10)
    at r.v (fileService.ts:592:15)
    at async v (resourceLoading.ts:68:18)
    at async s.rb (webviewElement.ts:733:19)
index.html?id=f67e5496-d36b-4c40-8dc1-9b25cfd6b174&origin=3a73e17a-8f8d-450c-8c89-b71d3bbdebf2&swVersion=4&extensionId=asciidoctor.asciidoctor-vscode&platform=electron&vscode-resource-base-authority=vscode-resource.vscode-cdn.net&parentOrigin=vscode-file%3A%2F%2Fvscode-app:999     GET https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/ed/antora/common.ad/doc/modules/ROOT/pages/same-in-english.jpg 404
(anonymous) @ index.html?id=f67e5496-d36b-4c40-8dc1-9b25cfd6b174&origin=3a73e17a-8f8d-450c-8c89-b71d3bbdebf2&swVersion=4&extensionId=asciidoctor.asciidoctor-vscode&platform=electron&vscode-resource-base-authority=vscode-resource.vscode-cdn.net&parentOrigin=vscode-file%3A%2F%2Fvscode-app:999
setTimeout (async)
onFrameLoaded @ index.html?id=f67e5496-d36b-4c40-8dc1-9b25cfd6b174&origin=3a73e17a-8f8d-450c-8c89-b71d3bbdebf2&swVersion=4&extensionId=asciidoctor.asciidoctor-vscode&platform=electron&vscode-resource-base-authority=vscode-resource.vscode-cdn.net&parentOrigin=vscode-file%3A%2F%2Fvscode-app:997
(anonymous) @ index.html?id=f67e5496-d36b-4c40-8dc1-9b25cfd6b174&origin=3a73e17a-8f8d-450c-8c89-b71d3bbdebf2&swVersion=4&extensionId=asciidoctor.asciidoctor-vscode&platform=electron&vscode-resource-base-authority=vscode-resource.vscode-cdn.net&parentOrigin=vscode-file%3A%2F%2Fvscode-app:1028

[ggrossetie]

It might be an issue with the directory name common.ad. In theory, the antora.yml file (file:///c%3A/Users/ed/antora/common.ad/doc/antora.yml) should be assigned to the AsciiDoc file file:///c%3A/Users/ed/antora/common.ad/doc/modules/ROOT/pages/goodbye.adoc.

Something must be wrong in: https://github.com/asciidoctor/asciidoctor-vscode/blob/15b4562d6e834518b35ad1b12e4943035cf0fac3/src/features/antora/antoraSupport.ts#L172-L189

[eddennison]

I changed the directory names to not include ".ad" but the test on line 182 is failing. Here's what I see in the debugger (and after adding a debug log message):

There is an extra slash (file:////c:/ vs file:///c:/) in the path from antoraConfig when compared with textDocumentUri.

[eddennison]

Also, I have a bunch of other questions about the code:

In order to support include directives for preview in the plug in, here's what I think is required:

• At document load time, need to search for and enumerate all Antora modules within all content source roots within the workspace directory. This code would be along the lines of findAntoraConfigFile() in antoraSupport.ts.
• This enumeration would need to generate a correspondence between the module name and a file system path to the module root directory (the directory with antora.yml).
• When include directives are encountered in the file being prepared for preview, the content must be loaded from either the current module or the specified module. The fs code loading the file must map from a module name to a specific fs path, then attempt to open the requested file.
• I believe that in order to actually open and read the file, the fs path needs to be sanitized to work with VS Code, specifically by passing it through asWebviewUri().

In the "normal" Antora execution path, the module name to fs correspondences are presumably generated when the playbook file is processed. Specifically, any module references are going to be limited to modules found in sources declared in the playbook file. And of course they're not really file system references but git references, since they include branch information.

Since the plugin code doesn't process playbooks, I assuming that we need specific code to search for modules in the workspace directory. I think this is what the IntelliJ plugin is doing. Can you give me any insights into where in the plugin code I should be looking?

[ggrossetie]

Also, I have a bunch of other questions about the code:

You can join the community chat: https://asciidoctor.zulipchat.com/#narrow/stream/406819-users.2Fasciidoctor-vscode

There is an extra slash (file:////c:/ vs file:///c:/) in the path from antoraConfig when compared with textDocumentUri.

The main issue is that, on your system, the moduleUri isn't really a URI. I guess we shouldn't use path.dirname / path.join but remove the last segment of the URI. Otherwise, we get \\ and it won't work.

[eddennison]

Also, I have a bunch of other questions about the code:

You can join the community chat: https://asciidoctor.zulipchat.com/#narrow/stream/406819-users.2Fasciidoctor-vscode

There is an extra slash (file:////c:/ vs file:///c:/) in the path from antoraConfig when compared with textDocumentUri.

The main issue is that, on your system, the moduleUri isn't really a URI. I guess we shouldn't use path.dirname / path.join but remove the last segment of the URI. Otherwise, we get \\ and it won't work.

Thank you. I can clean up the code so that it works on my Windows system, but obviously it needs testing on Linux. Also, I am guessing that since the returned antoraConfig is not a valid URI, it will also cause problems later in getResource().

[ggrossetie]

Probably, the goal is to use URI "everywhere" to avoid issues between Linux and Windows. To some extent it also helps with the Web portability.

Feel free to submit a pull request. I will try to run the test suite on Windows as part of the CI.

[ggrossetie]

Also, I have a bunch of other questions about the code: In order to support include directives for preview in the plug in, here's what I think is required:

This enumeration would need to generate a correspondence between the module name and a file system path to the module root directory (the directory with antora.yml).

We are leveraging the contentCatalog from Antora.

I believe that in order to actually open and read the file, the fs path needs to be sanitized to work with VS Code, specifically by passing it through asWebviewUri().

We will need to use an include processor (we can take a look at: https://gitlab.com/antora/antora/-/blob/main/packages/asciidoc-loader/lib/include/include-processor.js?ref_type=heads)

Since the plugin code doesn't process playbooks, I assuming that we need specific code to search for modules in the workspace directory. I think this is what the IntelliJ plugin is doing. Can you give me any insights into where in the plugin code I should be looking?

I don't think the Intellij plugin resolves modules outside of the workspace directory (or even read the playbook) but I might be wrong. Maybe @ahus1 can shed some light?

[mojavelinux]

I don't think the Intellij plugin resolves modules outside of the workspace directory

This is potentially where the manifest file produced by Antora Atlas comes into play. Here's an example of it in action: https://docs.couchbase.com/site-manifest.json

It's still a work in progress, but I think we should at least contemplate how information from that file may help with code assist.

[ggrossetie]

@mojavelinux do we need to look for a site-manifest.json at a specific location? should we make it configurable in the settings (i.e., path to the manifest)?

[mojavelinux]

It will be at the root of the published site (local or remote), though I do think it should be configurable since you don't necessary know what a full site build entails from the perspective of the tooling.