groff error on macOS-10.15.7

[bit-fu]

Cmdliner-1.1.0 apparently uses an invalid command line option to groff on macOS. When I use my tool in a way that should trigger help display, I get an empty man page display and the following error after hitting “q”:

groff: invalid option -- K
usage: groff [-abceghilpstvzCENRSUVXZ] [-Fdir] [-mname] [-Tdev] [-ffam]
       [-wname] [-Wname] [-Mdir] [-dcs] [-rcn] [-nnum] [-olist] [-Parg]
       [-Larg] [-Idir] [files...]
groff -h gives more help

This happens on macOS-10.15.7 (“Catalina”).

My tool uses subcommands. I was able to evade the above problem when calling the tool without a subcommand by using ~default:Term.(ret (const (`Help (`Plain, None)))) when applying Cmd.group, i.e. disable the use of groff and just print to stderr. A quick fix on my side could be to do the same for all subcommands, but I don't know how. As it stands, I need to do ./tool subcommand --help=plain to see subcommand documentation.

[dbuenzli]

Duplicate of #139. Indeed it seems that when we brought UTF-8 man page support in https://github.com/dbuenzli/cmdliner/commit/3fe9fcf5ea991bb3bf37117fd4c77fcfaca1b3d7 we didn't realize mandoc didn't ship on earlier macos version.

Please don't start working around in your own code. Any cmdliner program is affected at that point.

If you want to work around now you can do brew install mandoc.

Do you happen to know where I can find a list of which programs ship on a stock macos install of a given version ?

[bit-fu]

Re command list: I didn't find anything useful in the wild, so I put the question to Apple via my developer account. I'll relay their answer in this issue thread. It should arrive within two business days.

Re mandoc: Indeed, this is the simplest possible temporary fix, and I'm using it for now.

[bit-fu]

I've got an answer on the Developer Forum, but regrettably it's not encouraging. You should be able to read the Q&A here: https://developer.apple.com/forums//thread/702210?answerId=707277022

[dbuenzli]

Thanks anyways. But indeed it's a bit annoying we can't know how and when mandoc gets installed on macOS.

But then I bet you do have xcode command line tools installed and there's no /usr/bin/mandoc and you are on 10.15. Personally I'm on 11 with the latest xcode installed.

Do you have the latest xcode aswell ? Or at least the latest you can get on 10.15 ?

I'm not exactly sure what to do. A balance needs to be found between:

1. In my eyes it's not unreasonable to be on macOS 10.
2. Going back to non UTF-8 man page is out of the question, cmdliner's generated manpages now use the feature itself.
3. Doing the appropriate conversion in Cmdliner itself shouldn't be hard especially with 4.14's Stdlib UTF-8 decoders, but it won't be some time before 4.14 can be required, requiring 4.08 already annoyed the odoc people. Also according to that comment mandoc still gives better results.

[bit-fu]

I have the latest Xcode + command line tools for 10.15 installed (12.4), and there is no /usr/bin/mandoc. Is it there in macOS-11? I installed mandoc via MacPorts.

Maybe the best you can do for the time being is to leave a hint in the documentation that Mac users need to check for and/or install mandoc. If memory serves, I needed the MacPorts install of ocaml as the bootstrapping compiler for my very first opam switch anyway, so using that same package manager to also install mandoc would be as quick and easy for everyone concerned as it was for me.

[dbuenzli]

Is it there in macOS-11?

It is. So I guess it appeared in macOS 11. Maybe @MisterDA who looked into these things knows better ?

Maybe the best you can do for the time being is to leave a hint in the documentation that Mac users need to check for and/or install mandoc.

Yeah. Though I have to admit that was really not the idea. cmdliner should work out-of-the box on reasonably recent systems.

[MisterDA]

Thanks anyways. But indeed it's a bit annoying we can't know how and when mandoc gets installed on macOS.

Agreed. The person at the Apple support suggested:

[1] If I need to know whether a specific tool is available on a specific OS release, I spin up my VM for that release.

I tried to do that, but the VM wouldn't boot.

In my eyes it's not unreasonable to be on macOS 10. Yeah. Though I have to admit that was really not the idea. cmdliner should work out-of-the box on reasonably recent systems.

Agreed.

I tried looking at Apple Opensource archives (the archive's a joke), there's nothing to help us there.

Maybe @MisterDA who looked into these things knows better ? we didn't realize mandoc didn't ship on earlier macos version.

That's what I missed, I'm sorry for that. I thought it would have been shipped earlier. I don't see a way to process UTF-8 groff documents in macOS 10. Apparently macOS 11 ships mandoc-the-tool which correctly processes UTF-8 groff documents. The only way forward seems to do the preconv ourselves as you've noted. That will at least prevent the error and show the man page to some extent. And if groff -Tutf8 cannot even display UTF-8 characters from a document where we have carefully escaped to unicode code points all these nasty characters, well… (btw if it's still groff 1.19 on macOS 11 we may re-test to see how flawed it is, I've put up a small example) :skull:

[dbuenzli]

Not really happy about the decision, but I won't do anything about that.

In case people hit the problem the solution is to brew install mandoc. If you distribute a binary that users cmdliner and want to support pre-11 users then the solution is to make it depend on that package.