[Feature Request]: An improved method to resolve variable expressions

[mattcasters]

What would you like to happen?

Right now, if we have a variable expression, for example ${MY_VARIABLE}, any value that we have on the books for MY_VARIABLE in the local IVariables instance will be substituted and returned during IVariables.resolve().

The API that is used StringUtil.substitute() already supports recursive or nested variable resolution. However, this method is not used anywhere right now.

It would be great if we could give Hop users a way to signal that recursive or nested substitution of variables should take place. Some suggestions come to mind:

• A new way to specify variables, for example with round brackes $(MY_VARIABLE)
• A prefix system to the variables: ${NESTED(2):MY_VARIABLE}, for example to resolve the variable 2 levels deep.

If we think about the second prefixing system we can even think of a pluggable, configurable, system which would allow us to do more than just nested variables. For example we could build a ${ENCODED:DATABASE_USER}, for example if you want to obfuscate or encrypt the username of a database (not just its password) in an environment properties file.
We could combine prefixes, for example: ${ENCODED(EAS2):NESTED(1):MY_VARIABLE}. Other ideas are ways to get values from a secrets vault or resolve a value from a certain web-service.

The counter argument to all this is that it's a very cryptic way of doing things. Even as I'm typing this I'm not liking it. Perhaps we can think of a pluggable system attached to the lifecycle environment in which we're allowing these options for variables. That way we can build proper user interfaces reflected in proper configuration files and metadata.

Issue Priority

Priority: 3

Issue Component

Component: API