8325168: JShell should support Markdown comments

[lahodaj]

This is an attempt to add Markdown support in documentation comments to JShell.

It works by converting the Markdown text to HTML during the process of resolving {@inheritDoc} tags.

---

Progress

• [ ] Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• [x] Change must not contain extraneous whitespace
• [x] Commit message must refer to an issue

Issue

• JDK-8325168: JShell should support Markdown comments (Enhancement - P3)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/17686/head:pull/17686
$ git checkout pull/17686

Update a local copy of the PR:
$ git checkout pull/17686
$ git pull https://git.openjdk.org/jdk.git pull/17686/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 17686

View PR using the GUI difftool:
$ git pr show -t 17686

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/17686.diff

Webrev

Link to Webrev Comment

[bridgekeeper[bot]]

:wave: Welcome back jlahoda! A progress list of the required criteria for merging this PR into pr/16388 will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk[bot]]

@lahodaj The following labels will be automatically applied to this pull request:

• compiler
• javadoc

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

[jonathan-gibbons]

Compared to the previous PR, which was reasonably simple and quite elegant, this one seems unduly complicated and somewhat invasive. It is also using the transformer mechanism in a way that is not as intended.

The jdk.internal.md module is intended to be for JDK support for Markdown. As such, it has an imported copy of commonmark-java and now the new transformer support, which is just intended to be used to support any JDK extensions to Markdown, such as the new support for reference links for program elements. (There are suggestions for other future features in the JEP.) As such, I would expect all JDK clients to use the existing "standard" transformer, and to not substitute their own. Ideally, the "standard" transformer would be used by default, but I wanted to avoid an unconditional dependency from jdk.compiler to jdk.internal.md -- so hence the use of the service provider mechanism. (I guess I could update jdk.compiler to use requires static to establish a compile-time dependency on the jdk.internal.md module, and then try and install the standard transformer, which would always be available if we put a hard dependency in modules like jdk.javadoc and jdk.jshell.)

Your previous PR was pretty good (although I accept we did not get as far as a full review and approval.) The recent changes the overall Markdown work were made with that PR in mind. As such, the intention was that only two fairly small changes would be required.

1. Change the means to detect Markdown comments, from looking for /**md to using the new CommentKind support in DocTrees.
2. Install the 'standard' transformer, to enable the JDK extensions to Markdown

With those two changes, you can use the previous technique to parse and render a Markdown doc comment as before.

[lahodaj]

Thanks for the comments! I've dropped the ToHTML transformer, and made the transformations easier. Sadly, the standard transformer seems to break raw text tree positions, so it is not currently possible to properly replace the Markdown text with the Javadoc text when the transformer is installed. Please see: https://github.com/openjdk/jdk/pull/16388#discussion_r1479306878 for more details. Once that is solved, it should be possible to merge the newest changes in the Markdown branch.

[openjdk[bot]]

@lahodaj This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8325168: JShell should support Markdown comments

Reviewed-by: jjg

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 59 new commits pushed to the master branch:

• 9ee741d1e55c2520b28a5e3ca0604073d81d0059: 8332015: since-checker - Add @ since tags to jdk.httpserver
• 0f4154a9e9805534595feccc53a4a1abf20f99ae: 8331193: Return references when possible in GrowableArray
• 64bbae75121ccf80c02a0960e2db62eb558052e6: 8333394: C2: assert(bol->is_Opaque4() || bol->is_OpaqueInitializedAssertionPredicate()) failed: Opaque node of non-null-check or of Initialized Assertion Predicate
• c7495fb35d7736815d5777ab776ace013f9d50b5: 8333444: Parallel: Inline PSParallelCompact::mark_obj
• 454660d361e39f362ff0e10a5c2389af910cca23: 8332900: RISC-V: refactor nativeInst_riscv.cpp and macroAssembler_riscv.cpp
• 67d6f3ca9e8d1312c9e3a85dbe19903619f59064: 8332905: C2 SuperWord: bad AD file, with RotateRightV and first operand not a pack
• ca3072635215755766575b4eb70dc6267969a550: 8332866: Crash in ImageIO JPEG decoding when MEM_STATS in enabled
• 29e10e4582c1a844a6db4c42ba01bd1d6d4dfd52: 8332547: Unloaded signature classes in DirectMethodHandles
• c7d2a5c1c4e86955100f4c40170dc25222abd07f: 8314070: javax.print: Support IPP output-bin attribute extension
• d230b30353f59135287436b09949b80e9fd73a93: 8333398: Uncomment the commented test in test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarAPI.java
• ... and 49 more: https://git.openjdk.org/jdk/compare/2ab8ab56130ca258bf0347ea44e74a8cad3d537d...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[mlbridge[bot]]

Webrevs

• 05: Full - Incremental (64c984d6)
• 04: Full (d85a2b8a)
• 03: Full (4b02060d)
• 02: Full (9b9550f4)
• 01: Full (28211c75)
• 00: Full (0ba0fffd)

[bridgekeeper[bot]]

@lahodaj This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[openjdk-notifier[bot]]

The parent pull request that this pull request depends on has now been integrated and the target branch of this pull request has been updated. This means that changes from the dependent pull request can start to show up as belonging to this pull request, which may be confusing for reviewers. To remedy this situation, simply merge the latest changes from the new target branch into this pull request by running commands similar to these in the local repository for your personal fork:

git checkout JDK-8298405
git fetch https://git.openjdk.org/jdk.git master
git merge FETCH_HEAD
# if there are conflicts, follow the instructions given by git merge
git commit -m "Merge master"
git push

[openjdk[bot]]

@lahodaj this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout JDK-8298405
git fetch https://git.openjdk.org/jdk.git master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

[openjdk[bot]]

⚠️ @lahodaj This pull request contains merges that bring in commits not present in the target repository. Since this is not a "merge style" pull request, these changes will be squashed when this pull request in integrated. If this is your intention, then please ignore this message. If you want to preserve the commit structure, you must change the title of this pull request to Merge <project>:<branch> where <project> is the name of another project in the OpenJDK organization (for example Merge jdk:master).

[openjdk-notifier[bot]]

The parent pull request that this pull request depends on has now been integrated and the target branch of this pull request has been updated. This means that changes from the dependent pull request can start to show up as belonging to this pull request, which may be confusing for reviewers. To remedy this situation, simply merge the latest changes from the new target branch into this pull request by running commands similar to these in the local repository for your personal fork:

git checkout JDK-8298405
git fetch https://git.openjdk.org/jdk.git master
git merge FETCH_HEAD
# if there are conflicts, follow the instructions given by git merge
git commit -m "Merge master"
git push

[lahodaj]

/integrate

[openjdk[bot]]

Going to push as commit 8d3de45f4dfd60dc4e2f210cb0c085fcf6efb8e2. Since your change was applied there have been 59 commits pushed to the master branch:

• 9ee741d1e55c2520b28a5e3ca0604073d81d0059: 8332015: since-checker - Add @ since tags to jdk.httpserver
• 0f4154a9e9805534595feccc53a4a1abf20f99ae: 8331193: Return references when possible in GrowableArray
• 64bbae75121ccf80c02a0960e2db62eb558052e6: 8333394: C2: assert(bol->is_Opaque4() || bol->is_OpaqueInitializedAssertionPredicate()) failed: Opaque node of non-null-check or of Initialized Assertion Predicate
• c7495fb35d7736815d5777ab776ace013f9d50b5: 8333444: Parallel: Inline PSParallelCompact::mark_obj
• 454660d361e39f362ff0e10a5c2389af910cca23: 8332900: RISC-V: refactor nativeInst_riscv.cpp and macroAssembler_riscv.cpp
• 67d6f3ca9e8d1312c9e3a85dbe19903619f59064: 8332905: C2 SuperWord: bad AD file, with RotateRightV and first operand not a pack
• ca3072635215755766575b4eb70dc6267969a550: 8332866: Crash in ImageIO JPEG decoding when MEM_STATS in enabled
• 29e10e4582c1a844a6db4c42ba01bd1d6d4dfd52: 8332547: Unloaded signature classes in DirectMethodHandles
• c7d2a5c1c4e86955100f4c40170dc25222abd07f: 8314070: javax.print: Support IPP output-bin attribute extension
• d230b30353f59135287436b09949b80e9fd73a93: 8333398: Uncomment the commented test in test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarAPI.java
• ... and 49 more: https://git.openjdk.org/jdk/compare/2ab8ab56130ca258bf0347ea44e74a8cad3d537d...master

Your commit was automatically rebased without conflicts.

[openjdk[bot]]

@lahodaj Pushed as commit 8d3de45f4dfd60dc4e2f210cb0c085fcf6efb8e2.

:bulb: You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.