Question: is it possible to make CLI replace parameter dashes with underscores?

[iskandr]

I'm curious about replacing argparse with jsonargparse in some of my projects since simply running CLI(main_fn) would cover my most common use cases. However, it's not clear to me if there's any hook to change the mapping from Python argument names to CLI parameters. Specifically, I typically have multi-word parameters represented with dashes as commandline arguments. So, arg_like_this in Python comes --arg-like-this on the commandline. Is there any way to replicate this behavior with jsonargparse?

Thanks!

[mauvilsa]

arg_like_this in Python comes --arg-like-this on the commandline.

Currently no. But could be added.

[iskandr]

@mauvilsa I don't mind digging into the code if you have a hunch about the most harmonious direction to try. Should I add it as a specific option to CLI (eg convert_underscores_to_dashes=True) or make a more general transform function (eg commandline_name_transform=lambda x: x.replace("_", "-"))?

[mauvilsa]

@iskandr certainly, please dig into the code. Contributions are welcome! Though, note that this feature is not as simple as it sounds. New features only make sense if they play well and are consistent with all other existing features. It is not just modifying CLI. This needs to be implemented for ArgumentParser and be used in all cases: main parser, subparsers (a.k.a. subcommands) and dynamically generated parsers (type subclasses and callables). And make sense with respect to --help and --print_config.

There are questions to answer:

• If from command line dashes are used, how is it in config files?
• Would the dashes be in addition to underscores? If both, https://github.com/omni-us/jsonargparse/pull/255 would be required.

One important question is: what is the motivation for this feature and does it justify the effort to implement and maintain? I have thought about this feature several times, and the motivation mostly is preference and be slightly easier to type in most keyboards. On the other hand, a drawback of dashes is that the command line options are no longer exactly the same as the corresponding parameters in the source code. Not a very strong motivation, which is why I haven't implemented. @iskandr why exactly do you need it? What does it imply if you don't have it?

I have been refactoring and cleaning up the code to make it easier for people to contribute. Unfortunately, it is still not easy.

[iskandr]

Thanks for your reply @mauvilsa.

I can start with the motivation and circle back on implementation details once I get more familiar with this repo.

I would want hyphens/dashes (instead of underscores) to maintain consistency with other tools I have developed over the past decade. It would be very confusing to have to switch between dashes and underscores depending on the tool.

More generally, dashes are the GNU standard (https://www.gnu.org/software/libc/manual/html_node/Argument-Syntax.html) and (somewhat subjectively) seem to be preferred by many other library/tool maintainers (eg https://github.com/pytorch/pytorch/issues/65052, https://github.com/rspec/rspec-core/issues/497).

[iskandr]

How about:

If convert_underscores_to_hyphens=True (or some similar parameter name) then "_"->"-" happens for commandline forms of arguments (eg --multi-word-parameter) and thus show up in --help. However, the serialized config would always use underscores (eg `multi_word_parameter) and thus not require https://github.com/omni-us/jsonargparse/pull/255. @mauvilsa what do you think?

[mauvilsa]

Some thoughts:

• Given that the motivation is only to allow more standard command line behavior (hyphens), it does make sense that for config files, the original parameter names are used (underscores).
• The --help is not just to explain individual command line options, but also config files and environment variables. Thus, the help would need to show both with hyphens and underscores.
• Having both underscores and hyphens is required in cases where there is a need for backward compatibility, e.g. pytorch#65052.
• Implementation-wise it is not possible to have an argument that uses one name for command and another for the rest. Thus, the feature seems more viable as "add hyphens alias" instead of "replacing underscore with hyphens".
• The result of parsing is a Namespace object. This is used internally in jsonargparse for serializing configs (e.g. --print_config) and for instantiating classes. Both of these require original parameter names, thus in the namespace it would always be original names (with underscores, not hyphens).
• The first long option of an argument would always be with underscores, the second one would be the alias with hyphens.
• #255 is not required.

In the previous points when I say that something is not possible or be a certain way, does not mean that it is the only possibility. What I mean is that doing it differently will very likely complicate the code significantly. As I mentioned before, the code is already complex enough and I am trying to simplify things. Thus, please consider this as very important.

Is it for you reasonable the compromise that always the arguments have both options?

One possible improvement that could be done on top of this, is that for tab completion (argcomplete and shtab) only the hyphen options are completed.

I think it is now more clear to me how all of this could be implemented. Would you like me to describe it?

[iskandr]

I'm going to close the issue for now and try out defopt to see if it covers all of my relevant use-cases. It's a smaller library, so I might circle back here if I need the full functionality of jsonargparse and try to implement @mauvilsa's vision for making CLI-only hypen aliases.

(and yes, I think allowing both --arg-name and --arg_name on the commandline sounds fine to me, as long as there's a clear default)

[iskandr]

May revisit in the future!

[mauvilsa]

I will keep this open since other people might be interested.

[shaybensasson]

Hi, wonderful library.

I'm again emphasizing the need to support the conventional (I think) multi-word dash named args. This would help tremendously in migrating existing codebases.

IMO an alias on named-args in the CLI is enough (the default, the help, yaml should all be _).

Thanks, Shay

[noamsgl]

Hi, I'd like to see this feature as well.