Add the possibility to configure the CONFIG_FORCE_ prefix

[crash-g]

Description

When using config.override_with_env_vars, the library looks for env variables that are prefixed with CONFIG_FORCE_. Some applications may prefer to set a custom prefix instead, so this commit adds the possibility to customize it using the config.override_with_env_vars_prefix JVM variable.

Related issue.

What's left

I'd like to add some tests but for some reason some tests are failing locally (I've tried them before changing anything and they were still failing).

In particular, the ConfigTest:testLoadWithEnvSubstitutions is failing, which is the test I think I should use as reference to write a new test. The error is

java.lang.AssertionError: 
Expected :1
Actual   :42

	at org.junit.Assert.failNotEquals(Assert.java:743)
	at com.typesafe.config.impl.ConfigTest.testLoadWithEnvSubstitutions(ConfigTest.scala:1133)

Is there some test setup which is expected before running tests?

Other comments

I have noticed that the algorithm that matches environment variables with properties is case-sensitive, so CONFIG_FORCE_WHATEVER will not replace a property called whatever. I understand that it depends on the fact that whatever and WHATEVER are in fact different properties but I still think it's a little unexpected (at least coming from Spring).

Should this behavior be stated more clearly in the documentation?

[lightbend-cla-validator]

Hi @crash-g,

Thank you for your contribution! We really value the time you've taken to put this together.

Before we proceed with reviewing this pull request, please sign the Lightbend Contributors License Agreement:

https://www.lightbend.com/contribute/cla

[andreaTP]

Hi @crash-g ! Thanks for this contribution, I'm pretty positive about it, I think its time to introduce this feature!

I cannot reproduce your problem with the tests (and the CI used to confirm that they are passing, at least up to the latest release that I did), looks like you have something in the environment variables that are overriding that specific value ...

You are completely correct in taking as an example for the tests testLoadWithEnvSubstitutions, beware that those values are set by sbt: https://github.com/lightbend/config/blob/master/build.sbt#L88-L98

Regarding the comment on case sensitive variables, please go ahead and make it more explicit! 🙂

[andreaTP]

Quick heads up, the CI https://travis-ci.com/github/lightbend/config/jobs/439058491#L559 confirms that tests are running as expected on master

[crash-g]

I cannot reproduce your problem with the tests (and the CI used to confirm that they are passing, at least up to the latest release that I did), looks like you have something in the environment variables that are overriding that specific value ...

My bad, I'm not familiar with Scala, I was assuming the IntelliJ plugin would magically work but it does not. Anyway, I have now managed to execute all tests successfully using sbt directly.

However, I have now another problem. I have written the following test (and added CUSTOM_PREFIX_a_b_c to build.sbt)

@Test
def testLoadWithEnvSubstitutionsCustomPrefix(): Unit = {
    System.setProperty("config.override_with_env_vars", "true")
    System.setProperty("config.override_with_env_vars_prefix", "CUSTOM_PREFIX_")

    try {
        val loader02 = new TestClassLoader(this.getClass().getClassLoader(),
            Map("reference.conf" -> resourceFile("test02.conf").toURI.toURL()))

        val conf02 = withContextClassLoader(loader02) {
            ConfigFactory.load()
        }

        assertEquals(200, conf02.getInt("a.b.c"))
        assertEquals(260, conf02.getInt("a_c"))

        intercept[ConfigException.Missing] {
            conf02.getInt("CONFIG_FORCE_a_b_c")
            conf02.getInt("CUSTOM_PREFIX_a_b_c")
        }

    } finally {
        System.clearProperty("config.override_with_env_vars")
        System.clearProperty("config.override_with_env_vars_prefix")
    }
}

This fails, I think because the loadEnvVariablesOverrides is static so System.setProperty("config.override_with_env_vars_prefix", "CUSTOM_PREFIX_") happens too late. Any idea how I could work around this?

Just for reference the error is

[error] Test com.typesafe.config.impl.ConfigTest.testLoadWithEnvSubstitutionsCustomPrefix failed: expected:<200> but was:<2>, took 0.005 sec

where 2 is the value that CONFIG_FORCE_a_b_c sets.

[ekrich]

You add to build.sbt something like this. https://github.com/scala-native/scala-native/blob/master/build.sbt#L671-L678

[andreaTP]

@crash-g can you push your WIP to this or another branch to let me reproduce? Thanks!

[crash-g]

I have worked around the issue by calling ConfigFactory.invalidateCaches() before and after the test. Is this a good solution?

[andreaTP]

@crash-g the configuration is loaded in a different ClassLoader, the invalidateCache workaround should not be needed IIUC.

[crash-g]

However, the test does not pass without it. What should be changed? @andreaTP

[modo-lv]

Is this ever getting merged? We use this config library everywhere in our enterprise, but due to the prefix, we ended up writing an ugly wrapper-hack that modifies JVM's internal environment variable registry on startup, just so we could use plain environment variables without prefix. This would allow us to finally drop that mess.

[ennru]

We do not intend to extend the functionality of "Typesafe Config" further. See https://github.com/lightbend/config#maintained-by