Support for masking values when rendering

[gregsymons]

In my app, I'm exposing the combined configuration on a REST endpoint. For the most part, configuration.root.render() is good enough. However, I'd like to mask out sensitive keys that hold things like passwords. I've been able to implement masking like this:

val shouldMask = "(secret|password)".r.unanchored
def masked(value: ConfigValue) = {
   ConfigValueFactory.fromAnyRef("MASKED", "Value masked for security reasons")
}

def safeConfiguration(configuration: Config) = {
  configuration.entrySet.asScala.foldLeft(configuration) { 
    case (config, (key@shouldMask(blacklisted), value)) => {
      config.withValue(key, masked(value))
    }
    case (config, (key, _)) => {
      config
    }
  }
} 

The problem is that when I construct the new ConfigValue, I lose the original origin information. Ideally, I'd like to just add the "Value masked for security reasons" to the end of the existing comments, if any. However, there's no way to construct a new ConfigOrigin from the original origin, nor a way to pass one in if I could.

Really, this masking thing could be an additional ConfigRenderOptions option and hidden in render. Maybe you could do it like this:

def safeConfiguration(configuration: Config) = {
  val shouldMask = "(secret|password)".r.unanchored
  configuration.render(ConfigRenderOptions.defaults.withFilter {
    case (shouldMask(_), _) => true
    case _ => false
  }

Though that may be too Scala an interface for this library :smile:. An easier solution would be to just expose a way to "mutate" a ConfigOrigin and pass it into ConfigValueFactory.fromAnyRef so I can implement it myself without either making the "pretty" output of render ugly or losing the origin information. Or suggest a way I can do it with the existing API:)

[havocp]

Hmm. Trying to think of the way to enable this with the least API surface. It seems like the minimum is to allow specifying an origin when creating a ConfigValue with fromAnyRef etc. Then the elaboration would be to add a way to build a modified ConfigOrigin. I think those changes make more sense than special API for masking.

Another consideration though is whether masking should somehow be "built in" to the result of ConfigFactory.load already - akka (or maybe play) for example has an option to log the whole config on startup IIRC - so then you'd want that logging by another library to also mask... the simplest answer could be that "secret" and "password" (hardcoded, or maybe a regex found in the config itself) get auto-masked...

So that would be one option with no API addition at all, something like a config.maskRegex which could be secret|password by default, and when we render we look up config.maskRegex in the config itself, and apply it...

Seems like it's worth solving this problem somehow but I'm not sure yet what my favorite answer is.

[shanielh]

@havocp - Can we set another way to quote values (i.e, p"myPassword" for passwords), I've noticed that when rendering the configuraiton - We're able to see if the value is quoted or not.

[viktorklang]

My recommendation: don't put passwords in config files and don't rely on config masking to "secure" anything.

[clehene]

@viktorklang so use config for everything but for secrets use something else. Then why not use that something else for everything?

This question is not about storing secrets in a config file, but rather having it passed through a config object. As Typesafe config handles env variables integration (yet still displays them freely) it would be the normal mechanism to get the secrets in, provided it would offer the masking capability.

[viktorklang]

Since the Config objects are immutable (and do not use any access control), and secrets typically need sophisticated handling (invalidation, updates, cert management etc) I personally believe that Secret Management is superficially related to config management.

[clehene]

@viktorklang fair enough. Although secrets management may have a different lifecycle (and more complex) it's still a config. Any change could be handled by the config library if it can react to changes, or otherwise by the scheduling system (which could restart it)

One way to solve this is to configure a path to a file that contains the secret. E.g. in Kubernetes you can mount secrets inside a container. This would solve this concern as well.

[eneveu]

Would really love this too. I'd like to print the Config at application startup for debugging purposes, but I need to sanitize secrets.

Right now, I am doing this:

import java.util.Map.Entry

import scala.collection.JavaConverters.iterableAsScalaIterableConverter

import com.typesafe.config.{ Config, ConfigValue, ConfigValueFactory, ConfigValueType }

class ConfigRenderer(sanitizer: ConfigSanitizer) {

  def render(config: Config): String = {
    sanitize(config).root().render()
  }

  def sanitize(config: Config): Config = {
    var result = config

    for (entry: Entry[String, ConfigValue] ← config.entrySet().asScala) {
      val sanitizedValue: Option[String] = sanitizeValueIfNecessary(entry.getKey, entry.getValue)
      sanitizedValue.foreach { sanitized ⇒
        // TODO find a way to avoid losing origin information and comments when sanitizing (see https://github.com/lightbend/config/issues/145)
        result = result.withValue(entry.getKey, ConfigValueFactory.fromAnyRef(sanitized, "Masked for security reasons."))
      }
    }
    result
  }

  private def sanitizeValueIfNecessary(key: String, value: ConfigValue): Option[String] = {
    if (value.valueType() == ConfigValueType.STRING) {
      sanitizer.sanitize(key, value.unwrapped().asInstanceOf[String])
    } else {
      None
    }
  }

}

With a Sanitizer inspired by Spring Boot's Sanitizer, except I return an Option[String], to know if the value was sanitized... If it wasn't sanitized, I don't want to call .withValue(), since it would lose origin info / comments.

So instead of public Object sanitize(String key, Object value), I have:

  /**
   * Sanitize the given value if necessary.
   *
   * @return the sanitized value, or None if unchanged
   */
  def sanitize(key: String, value: String): Option[String]

If there was a way to construct a new ConfigValue while passing the origin and comments of the previous value, it would be perfect.