Documenting CLIs

[epage]

(forked from #8)

This is for discussing documentation formats and authoring.

[epage]

@killercup :

• documentation of env vars a CLI app repsects/uses
  • auto-generate from something like https://docs.rs/configure?
  • how to incorporate external crates, like env_logger?
• there seems to be no cohesive answer for man pages yet
  • ripgrep is using aciidoc: https://github.com/BurntSushi/ripgrep/pull/776
  • issue in clap https://github.com/kbknapp/clap-rs/issues/1037(@kbknapp, awaiting clap v3-alpha)

(edited to include the correct issue as pointed out by @kbknapp )

[Screwtapello]

Whatever system is in place for generating manpages, I hope it will be possible for a crate to include a handwritten manpage instead. Tools that convert other formats into manpages typically generate "man" format documentation, where "mdoc" uses more-semantic markup, and hence produces more professional-looking output.

Also, it's always going to be difficult to add a dependency on a tool that's not written in Rust. Python's docutils can convert ReStructuredText to man format, but then you've got to deal with lugging a Python interpreter around. Pandoc can convert just about anything to anything, but it's written in Haskell. As for Asciidoc... well, if I ask Debian to install it, it drags in 600MB of TeX, which seems like massive overkill to just format a manpage.

[BurntSushi]

I think relying on external tools to generate man pages is reasonable. If and when someone gets the inclination to write a tool in Rust that converts some human readable markup language to roff, then folks can use that. This is a build time tool, and it can be an optional build step that doesn't execute if the required tooling isn't present.

I agree that being able to hand write prose in the man page is an important requirement. The way I do it today is create a template that is hand written, but is filled in with documentation for each flag automatically at build time.

[kbknapp]

I agree with this issue, from the survey it looks like people really want a standard way to document their CLI applications. Some request manpage generation by name, others just say "document." Personally, I think manpages should be the priority because it's more complete and we're closing to achieving that goal. Once we're there, we can look at the other half of the spectrum and find a way to better document usage of the application (and less reference style like manpages) for those applications that wish to have it.

I've been ticking off the clap v3 checklist getting towards manpage generation, however I think it'd be a great idea to have a more general solution as well because not everyone wants to use clap, which is totally understandable. So an external tool is almost a necessity.

Perhaps step one, prior to finding/building an external tool is just getting cargo to recognize manpages (handwritten, or not) in the same way it recognizes READMEs?

[pickfire]

It's true that man pages can be generated but the main aim of having man pages is to have a more complete documentation that covers as many stuff as possible. The traditional CLI application help:

1. -h: Short and concise help
2. --help: GNU style long help that includes long help information with default arguments
3. man: Tiny details of application, most includes DESCRIPTION, SYNOPSIS, ENVIRONMENT, FILES, EXAMPLES, SEE ALSO and even help for config file.
4. info: I don't usually look at it, it's like a book. (preferably not having it)

But if we were to combine 2 and 3, we might loss many information. How should we tackle the issue of having the generate the information while being able to show as much as we can in 3?

[BurntSushi]

@pickfire Did you look at what ripgrep does? I write out the part of the man page I want to curate manually, but generate pieces of it (like the docs for every flag) automatically from my clap definition.

[pickfire]

@BurntSushi Yes, ripgrep man pages are indeed awesome though the --help feels too long-winded for me though. My suggestion to improve the man pages are adding .B (bold) to the options inside each each options' description.

For example in -x: This overrides the --word-regexp flag. Can be: This overrides the --word-regexp flag.

[BurntSushi]

OK, yeah that wasn't clear from your previous comment. You're talking about very specific formatting details of the man page. Yes, I agree, that is difficult in the way I did things for ripgrep. If I wanted to solve it, I'd probably write my help strings in Asciidoc (or whatever), and strip the markup before feeding it to clap, but let it flow through to man page generation. I haven't care enough to do such things yet. :-)

[ayosec]

https://github.com/rust-lang-nursery/cli-wg/issues/23#issuecomment-375329283

Also, it's always going to be difficult to add a dependency on a tool that's not written in Rust. Python's docutils can convert ReStructuredText to man format, but then you've got to deal with lugging a Python interpreter around. Pandoc can convert just about anything to anything, but it's written in Haskell. As for Asciidoc... well, if I ask Debian to install it, it drags in 600MB of TeX, which seems like massive overkill to just format a manpage.

cmark (which is available as a Debian package) can convert a Markdown file to the man format. The only missing piece is the .TH header.

[yoshuawuyts]

Opened an issue on Clap to extract information for man page generation. https://github.com/kbknapp/clap-rs/issues/1285

It's currently already possible in 2 ways, but I want to make sure that there's an approach that won't break overnight (currently the most reliable version relies on undocumented internals).

Hopefully we'll be able to hop on the man page generation train using Clap :grin:

edit: I'm looking into using troff-rs to generate the man pages. Perhaps that might be a good place to collaborate on better Man page generation, even for folks that don't necessarily use Clap :sparkles:

[yoshuawuyts]

Update on progress:

• https://github.com/rust-clique/man now exists to generate man pages (waiting on PR to be merged)
• https://github.com/yoshuawuyts/markdown-edit now exists to take any string, and update a markdown file with the content. Especially useful to insert a ## usage section with the CLI output I reckon :D
• we're waiting on Clap v3 to have a cohesive story between all these pieces, which should happen around this fall.

Hopefully together these should provide a reasonable documentation story!