Reading option values from environment variables

[RunningOstrich]

Can I specify an environment variable to use as a source for an option's value if the option was not specified on the command line?

The simplest way I have found so far is:

new Option<string?>("--some-option", () => Environment.GetEnvironmentVariable("SOME_OPTION"), "description...")

However, this runs into a couple of hurdles:

• The environment variable is expanded and shown in the help. This might be an issue if the environment variable is being used on sensitive options like --password, as typing in a command incorrectly could print the option to standard out.
• The name of the environment variable isn't printed in the help unless it is manually added to the description.

It would be nice if I could do something like:

new Option<string>("--some-option", () => "default-value", "description...") { 
  EnvironmentVariable = "SOME_OPTION", //Perhaps this could be an array instead?
};

The help builder could then present the environment variable information in a consistent & nice way like it does for defaults.

The issue with this is that it seems like the parser reading environment variables might be undesirable. An alternative may be to increase customization around defaults to let us print out default info differently with the existing help?

[bartr]

We did a PoC on this a few months ago - sounds like we should finish and submit a PR ...

Here is what we learned:

An issue we ran into is we want to have command line, env var and default values (in that order). Others suggested they might want to use config files or a config service or a database too / instead and perhaps in different orders. We need a "chain".

The approach we came up with during the PoC was to have an extension for env vars and give devs the ability to easily build (and share) other extensions.

Precedence was challenging but if your extensions have a "clear" method, then you can control your precedence by the order you add the extensions. For us, it would be new Option().Clear().UseCommandLine().UseEnvVar().UseMyExtension().UseDefault() which is clean and extensible.

If everyone is OK with command line always being first and default always being last, we could probably follow the AddValidator() approach and have AddValue() extensions.

A default function is challenging because you have to throw an exception if the env var is set but doesn't parse.

We temporarily worked around this by building ParseString, ParseInt, ParseBool, ParseStringList, etc. functions. Within the function, we check the env var and use it if it exists. We display a friendly error if the env var exists but doesn't parse.

The downside of this approach is

• you end up rewriting all the parsing code that's already in System.CommandLine
• you have to have a default value
• you can't declaratively supply a default value when you add the Option
• you have to set isDefault true when you create the option or the parser won't get called if the command line param is missing (which defeats the purpose for us)

Although not ideal, it's working well for us on several apps and doesn't have the potential to expose secrets on --help.

A better approach might be to add a validator that does this outside of parse. At the time, we didn't know how to determine if it was a passed value or a default value. That removes most of the downsides of our current approach. I'll test that approach ...

[Pxtl]

Yeah, I've done the same, but with a different filestore for fetching arguments from instead of the registry. Basically I added a custom AddOption<T> wrapper that fetches the description from the XmlComments, pulls the default value from the config file, and then puts it all together to create the Option. Works really well, and the defaults appear in --help mode.

I didn't find I had to mess with the parser at all... the built-in parsers seemed to do the job just fine. Surprised my experience is so different from yours.

[qmfrederik]

This would be a very useful feature to have. We allow users to specify option values on the command line, as environment variables (e.g. MY_OPTION_NAME=foo), or as files (e.g. /etc/product/my_option_name would just have contents foo, with optionally a \n at the end). It would be nice if this were built in the product and if these "fallback sources" were easily discoverable for users.

[bartr]

the AddOption<T> was brilliant - really simplified our code - THANK YOU!

our scenario is like the above where we have (in order) - default, env var, cmd arg

a simple test for --timeout int is to set TIMEOUT=foo

Since foo isn't an int, it won't parse and will throw an exception if you try to assign it (either in the parser or as a default value). We want the happy experience that you would get if you used --timeout foo on the command line (i.e. the red error and help display)

What I ended up doing is creating a List of parse errors and in my final validation, I add them. Works really well and the code is much cleaner.

LMK if there's a way in the default to get system.commandline to handle for you.

It would be trivial to add support for files or other data sources.

Note we still have the problem where --help will display the env var values. We're good with that because we don't pass secrets via env vars. If you do, it's a potential risk of exposure.

Were not parsing lists or files (yet) - here's the code (LMK if it can be improved)

if (defaultValue.GetType().IsEnum)
{
    if (Enum.TryParse(defaultValue.GetType(), env, true, out object result))
    {
        value = (T)result;
    }
    else
    {
        EnvVarErrors.Add($"Environment variable {key} is invalid");
    }
}
else
{
    try
    {
        value = (T)Convert.ChangeType(env, typeof(T));
    }
    catch
    {
        EnvVarErrors.Add($"Environment variable {key} is invalid");
    }
}

[attilah]

@bartr could you share the POC code in a partial PR or in gist or something as it has been 2.5 years now and would be nice to have something without duplicating the work you've already done.

Is there an update on having it shipped as a feature?

[beeradmoore]

Something else that would be nice to do with this, is if you specify an Option as required using an environment variable should consider that as used.

eg.

var rootCommand = new RootCommand("MyApp");
var inputOption = new Option<string>(new string[] { "--input", "-i" }, () =>
{
    var envValue = Environment.GetEnvironmentVariable("MY_APP_INPUT") ?? String.Empty;
    if (String.IsNullOrEmpty(envValue) == false)
    {
        return envValue;
    }
    return String.Empty;
}, "Some description");
inputOption.IsRequired = true;
rootCommand.AddOption(inputOption);
rootCommand.SetHandler(RunAsync, inputOption);

In this case: MyApp --input test is valid MyApp will report an error that input is required MY_APP_INPUT=test MyApp will report an error that input is required but we defined it as an environment variable so this should be allowed.