config
config copied to clipboard
lightbend
how to work with empty string
I have a configuration entry, which is supposed to be a number, that is passed from Openshift (as env var) and it's impossible to use unset variable in that configuration (it's yaml file produced by template engine). In case if configuration is missing - it would be empty string.
Docs say
env variables set to the empty string are kept as such (set to empty string, rather than undefined)
So this library passes it down as an empty string and then in runtime it fails on getInt with this error: "max-request-size-mb has type STRING rather than NUMBER". Is there any way to treat empty strings as a missing value, so that following will work as expected and use number 20, when STREAMS_PRODUCER_MAX_REQ_SIZE_MB is set to empty string?
max-request-size-mb = 20
max-request-size-mb = ${?STREAMS_PRODUCER_MAX_REQ_SIZE_MB}
Could that logic probably be changed to work differently with numbers? Treating empty string as an empty string, when number is expected makes no sense IMHO.
I don't think there's a way currently. If empty string were always treated as unset there would be no way to set things to empty string so it doesn't seem easy to change.
Not always, but maybe some way to configure it? Also it would make sense to behave that way by default when number is expected.
At the point you're doing the getInt the 20 has already been discarded unfortunately.
Since it would be global to the config not specific to this path, I'm not sure what the config option would be or at what point it would be set.
But why is it discarded?
When I declare max-request-size-mb = 20 it is already clear that value is a number. Why following line max-request-size-mb = ${?STREAMS_PRODUCER_MAX_REQ_SIZE_MB} cannot just ignore empty string?
@strowk Since this is an environment-induced problem, can't you change your launch script to clear the env vars which are set to ""?
I unfortunately do not control that launch script. Only can write template mapping env variables to some yaml config. So env variable would always be present, but it will be empty when yaml does not have value. Maybe there is some tricky way to exclude config, but personally I think that logic 'empty string->default value' seems to be better placed at this library (where we have defaults) instead of spreading it over several places (like 'empty string -> unset var' and then later 'unset var -> defaul value').
As to possible syntax for config.. well it could look like this f.e.
max-request-size-mb = 20
max-request-size-mb = ${??STREAMS_PRODUCER_MAX_REQ_SIZE_MB}
Two question marks mean that empty string would be considered as missing value as well.
I realize that there is a piece of information that is lost when our templating thingy produced empty string in place of something that was initially missing, but unfortunately a lot of ways to deploy stuff in cloud works this way and even when it's possible to not set env variable, it usually requires some more effort (like with helm, you'd have to wrap couple of lines in {{ if }} or something like that). That is why I think that community in general would benefit from having an option to ignore empty string here as well as unset variable is ignored with one additional symbol (like in example above, or maybe something different, that's just an idea)
My point-of-view is that the configuration shouldn't have to deal with all the ways that potential production of env-vars could be formulated and passed in—i.e. the logic to translate should live outside of the configuration: hence the suggestion to inject that logic in the environment which creates the problem, by running a script beforehand which converts misconstructed env-vars.
That unfortunately is not always possible or easy to do. It would also make overall code a bit more complicated. I realize that feature to treat empty strings as missing would make this library more complicated too, but it should not be very hard to cover with tests and if done here once, would help a lot of people with the same problem in different other environments.
And how would you be able to override config values with an empty string in that case? Introducing even more parsing rules seems to me like solving the wrong problem in the wrong place?
It's not always necessary to override config values with an empty string though. Particulary in my situation, when variable is an integer and empty string just does not make sense. I'm sure that one more parsing rule is exactly the right way to solve it.
