Flag priority; supporting config files overlapping with flags

[johnSchnake]

One reoccurring issue I have run into is the merging of config files and command line flags. I know that viper solves some of these problems, but it also introduces additional complexity to the developer and makes it a bit more awkward to setup in many, simple cases.

So if I could have something like this:

type configFile struct{
  value string
}

func main() {
  var cfg configFile

  // cfg is a bound config file
  pflag.Var( &cfg, ... ) // Assume cfg implements the necessary methods

  // binding a string value to a property of the given config
  pflag.StringVar( &cfg.value ... )

  // Parse the flags and know that the explicit string value will have precedence over the config value
  // regardless of their order on the cmdline and that that the default value for the StringVar flag wont
  // override the config.
  pflag.Parse()

  // Will be the explicit string value if set,
  // will be the explicit value from the config file otherwise (if set),
  // will be the default value [from some predictable source; not sure which makes most sense]
  fmt.Println(cfg.value)
}

I routinely run into situations with CLIs where there are many command-line options; some, or all of them, coincide with a config object which may also be passed via command-line options.

Currently, the only way to reconcile this is:

• awkward, manual logic to merge the two (which requires not just the values but the pflag objects themselves so that you can check whether or not values were set or are just default values)
• using extra viper commands to bind and get values; which gets increasingly awkward when the value for the config file is itself from a command line value

To facilitate this it seems like we need two things:

• a way to prioritize flags so that, regardless of their order on the cmdline they are set in a predictable order (so that the config file values won't overwrite explicitly set cmdline options)
• a simpler way to specify flags without default values (so the default cmdline option wouldn't overwrite the config file value)

Let me know what you think.

[jayunit100]

@hanlins and other folks in the sig-network community are talking about this as a solution to the overall confusion around kube proxy flag configuration confusion that people are facing

[hanlins]

I think we could potentially add some feature to flagSet to make it either selectively overwrite the variables based on whether the user has explicitly set the flag, or fail the Parse operation if there's a conflict on the variable and flag value. I can start a changeset for review.

[hanlins]

Our use problem in k8s is we take configurations from multiple places, one is a configuration file, and the other is the flags. And the tricky part is, the path for the configuration file is also being passed via flag. The current status is we're doing one-pass parsing for all the flags, then the flags are being filled into a struct object(obj1), and we unmarshal the config file into another object(obj2) of the same type, then we overwrite the obj1 with the values of obj2. So people are getting confused as the flags parameters they pass are overwritten so it's like a noop, super confusing. What I tried previously is to overwrite obj2 with obj1 and make sure the flag values take precedence, but there's another problem that if user hasn't set the flag, then we will overwrite the configurations specified in the config with the flag's default value, it also not acceptable. Given those issues we're facing, I think maybe we can introduce some new options to flagset/flag, so that when we pass the flag value to object that store it, we can optionly fail this overwrite(on conflict) or just not overwrite it if the flag value is from default.

[cornfeedhobo]

I'm fairly confused by this. Doesn't viper solve all these problems?