Output of --help with too much information

[harimkang]

🚀 Feature request

Hierarchy-level --help format:

There are many advantages to jsonargparse, and I'm a big fan of it. However, jsonargparse also has its drawbacks, and I believe one of them is the --help message format. So I think it would be nice to have a helpformatter(..?) that supports a simpler format and a verbose helpformatter that shows more details. --help --verbose (or --help_detail..?)

Motivation

I'm a big fan of jsonargparse, as I've said before, and I think this project will only grow. Thank you for creating this library. On the team, I'm working on adapting that library. I think jsonargparse has a lot to offer to developers like me, with its API and CLI harmonization and flexible features.

However, if a class or method with many parameters and a long docstring is registered via jsonargparse, the user will be overwhelmed with too much --help output. (This makes '--help' too informative for open community users, and actually hinders help.)

I think this is an easy issue to encounter with jsonargparse. (This is because it is closely tied to the API.) I thought it would be nice to have the ability to distinguish between --help messages that summarize the essentials and --help messages that go into more detail.

I'm hoping to get some ideas from the community, including repo maintainers, on this.

Pitch

For example, we support tons of hooks, but user don't need to know them all. (The above output will make it difficult for light users to use.)

I'd like to keep this, but also have the ability to show shorter, more concise --help output. I don't have any immediate suggestions, just that it would be nice to have a more customizable help formatter.

Alternatives

I've been thinking about disabling help altogether and providing it as a subcommand in the CLI. However, this would require quite a bit of effort and may not work well.

[mauvilsa]

I agree that the help can have too much information. And the idea of multiple levels of details is an interesting one.

Currently the formatter class is a bit messy and not easy to extend to customize the formatting. I have thought of cleaning it up and decoupling the part about config comments. The cleanup might be convenient to implement something like multiple levels of details in the help. Though, most likely this would be a breaking change in the formatter, so would only happen for jsonargparse v5.

[harimkang]

I agree that the help can have too much information. And the idea of multiple levels of details is an interesting one.

Currently the formatter class is a bit messy and not easy to extend to customize the formatting. I have thought of cleaning it up and decoupling the part about config comments. The cleanup might be convenient to implement something like multiple levels of details in the help. Though, most likely this would be a breaking change in the formatter, so would only happen for jsonargparse v5.

Thank you for your response. I'm very much looking forward to future updates. For now, I've solved this in my own simple way - it's still messy and requires refactoring, but I got it to work in a tricky way.

This is done by overriding the helpformatter class, defining a variable called verbose_level in that class and determining the output based on that. This is how we filter by verbose_level in add_usage and add_argument. (We need to define what we want to filter for, but for now i've run it and defined what command absolutely need so that only that is exposed).

(There's a link to an example below, but note that this is PoC-level code that hasn't been released yet and is subject to change.) https://github.com/openvinotoolkit/training_extensions/blob/c27d73a57cb2b04b12f61210fe86255a8d0234dd/src/otx/v2/cli/utils/help_formatter.py#L105

I'll be sure to share this here we've adapted and released it.

[harimkang]

Hi. To counter this issue, I came up with a help formatter that varies the output based on the verbose level. I hope someone can use this method to solve the same issue, or come up with a better way.

We are implementing a training framework and training includes a lot of configuration of data, model, training parameters, etc. all configuration:

This is very desirable because all values are configurable, but it can create a perception of complexity for first-time users.

So we define 3 levels of verbose, and modify the help output to output in 3 levels. 1-level (default): Here we provide a quick guide for simple use.

otx train --help

This can make a very good impression on first-time users.

2-level: Provide only required argumentation info. For each subcommand, we declare only the required arguments and output information about them.

otx train --help -v

3-level: Print all configurable arguments.

otx train --help -vv

This shows the output of a traditional jsonargparse.

https://github.com/openvinotoolkit/training_extensions/blob/develop/src/otx/cli/utils/help_formatter.py Above is the HelpFormatter class we reconfigured.

If you have better suggestions or questions, please leave them. Thank you.

[mauvilsa]

Thank you so much for sharing!