spring-boot
spring-boot copied to clipboard
Support .env file for configuration properties
Lots of developers use .env to process own configuration for script or app. Now I add spring.config.import
in application.properties as following:
spring.config.import=optional:file:.env
and SpringBoot app throws the following exception:
Caused by: java.lang.IllegalStateException: File extension is not known to any PropertySourceLoader. If the location is meant to reference a directory, it must end in '/'
at org.springframework.boot.context.config.StandardConfigDataLocationResolver.getReferencesForFile(StandardConfigDataLocationResolver.java:199)
at org.springframework.boot.context.config.StandardConfigDataLocationResolver.getReferences(StandardConfigDataLocationResolver.java:121)
I think .env file could be considered as properties file for spring.config.import
.
@linux-china Thanks for getting in touch. Simply considering a .env
file as a properties file wouldn't do what I think most users expect.
When Spring Boot reads key/value pairs from the system environment, additional processing is required to map the typical UPPERCASE_WITH_UNDERSCORES
style of environment variable keys to the canonical form of Spring Boot property keys. This mapping is not done when configuration is read directly from a properties file. I suspect most users would use the environment variable style of keys in a .env
file, so the fact that the key name mapping is not performed when importing the file as a properties file would result in the expected properties not being present in the Spring Environment.
In order for this to work as expected, a .env
file would need to be recognized as a file containing key/value pairs similar to system environment variables, and processed accordingly. This is, of course, a bigger change than what you've suggested.
Can you elaborate on the types of properties that you would find convenient to set in a .env
file instead of a local application.properties
file?
@scottfrederick Now I use jbang to write some scripts for Spring Boot App, scripts and app share some same configuration, such as database url etc, and https://github.com/cdimascio/dotenv-java is very friendly for jbang and kscript. I also have some script managed by just command runner, and .env is also supported by oh-my-zsh dotenv plugin, and .env variables will be loaded into env variables automatically for some command line tools. I mean .env is convenient for developers.
But now still ok for me, I can use ln -s .env env.properties
and spring.config.import=file:env.properties
also works well. Or ln -s .env application.properties
for a local application.properties for Spring Boot.
@linux-china Here's a more concrete example of what I think would be the problem with loading a .env
file as a properties file:
In a properties file, you can set a property like spring.application.name=myapp
. If you want to set this property as an environment variable, you can set SPRING_APPLICATION_NAME=myapp
. Either property will resolve to spring.application.name
in the Spring Environment (e.g. using @Value(${spring.application.name})
).
If you have SPRING_APPLICATION_NAME=myapp
in a properties file, this will not be resolved to spring.application.name
because the property key mapping logic is not applied when reading properties files. Properties file are expected to use the canonical form.
Since .env
files typically contain keys that conform to the environment variable naming conventions, I think users would naturally set properties like SPRING_APPLICATION_NAME=myapp
in the .env
file and then be surprised when the application name wasn't set as expected (because the file is being read as a properties file and the property key mapping isn't done). This will work if you set spring.application.name=myapp
in the .env
file, but you'd have to understand how Spring Boot works to know to use that form.
I can use
ln -s .env env.properties
andspring.config.import=file:env.properties
In your use case, are you using the canonical form in .env
, or how are you setting the properties there?
yes, I use normal key style form in .env file, and my .env file content as following:
nick=linux_china
management.server.port=9999
then I create link for .env with ln -s .env env.properties
my application.properties as following:
spring.application.name=spring-boot24-demo
spring.config.import=optional:file:env.properties
All work well, @Value("${nick}") private String nick;
and management listen port is 9999
but one problem is the env variable, management.server.port
could not be used as env variable name, and you have described this problem. Maybe I should do some modification for dotenv plugin, and convert management.server.port
to MANAGEMENT_SERVER_PORT
automatically.
We think there is value in supporting .env
files as an enhancement to Spring Boot. We've marked the issue pending-design-work
to consider a few implementation options:
- Automatically process files named
.env
in the current directory - Add a new
ConfigDataLocationResolver
andConfigDataLoader
to support loading files with something likespring.config.import=envfile:.env
(note that this would read the contents of the file as a set of environment variables, not as a properties file as originally suggested) - Add support for a new extensionless-file hint type for files containing environment variable key-value pairs with something like
spring.config.import=file:/etc/config/.env[.env]
@scottfrederick I mad a mistake about .env file format, and key should be uppercase like following:
KEY=xxx
DATABASE_URL=xxxxx
Some frameworks use multi .env files like Spring Boot profile, for example Vue.js CLI.
.env
.env.local
.env.[mode]
I'm not sure some developers use following style or not.
.env
local.env
xxxx.env
Maybe Spring Boot should support to load .env.dev
or dev.env
styles by profile name.
I think maybe the better approach would be to parse the .env file upon application startup and provide the content as "virtual" environment variables, whose keys could be addressed in application.properties by regular placeholders. As such:
.env file content: DB_USER=mockUserName
application.properties content: spring.datasource.hikari.username=${DB_USER}
If anyone's still interested in using .env
as another properties file, for me it works by adding
# Development properties (also used for docker-compose)
spring.config.import=optional:file:.env[.properties]
into application.properties
.
The [.properties]
is a hint for spring which translates to "Use this file without an extension as a .properties
file."
If anyone's still interested in using
.env
as another properties file, for me it works by adding# Development properties (also used for docker-compose) spring.config.import=optional:file:.env[.properties]
into
application.properties
.The
[.properties]
is a hint for spring which translates to "Use this file without an extension as a.properties
file."
It works, but still some work for Spring Boot. Spring Boot should consider additional processing is required to map the typical UPPERCASE_WITH_UNDERSCORES style of environment variable keys in .env file.
One consideration that always comes up is that environment variables will impact all profiles, and while this sort of makes sense, it's sometimes tricky when running tests and things like that.
I think the implementation should take into consideration a way to set some environment variables for a specific profile — something like:
#
#
the_boot.ENV_VAR = value # ...will only impact the_boot profile
ENV_VAR_ALL = world # ...will apply at application level (like a usual OS environment variable declared and accessible anywhere)
Not trying to disrupt anyone else's work here, but there's a package for that: https://search.maven.org/artifact/me.paulschwarz/spring-dotenv/2.3.0/jar
I think maybe the better approach would be to parse the .env file upon application startup and provide the content as "virtual" environment variables, whose keys could be addressed in application.properties by regular placeholders. As such:
.env file content: DB_USER=mockUserName
application.properties content: spring.datasource.hikari.username=${DB_USER}
How do i "parse the .env file upon application startup and provide the content as "virtual" environment variables" So that i can read the values inside .env and use them on my application.properties file, i want to store my password on the .env than directly on the application.properties file. Please give me pointers. thanks
The co.uzzu.dotenv.gradle
Gradle plugin does that (in that context). If you need to read the environment variable values in the source code, you would need to use a library: io.github.cdimascio:dotenv-java
.
I hope that Spring Boot, at some point, unifies all of that "natively".
Thank you so much. However maven seems to not be able to find the co.uzzu.dotenv.gradle from the repos. I am getting the following error
co.uzzu.dotenv.gradle:co.uzzu.dotenv.gradle.gradle.plugin:pom:1.1.0 was not found in https://repo.maven.apache.org/maven2
when trying both:
```
<dependency>
<groupId>co.uzzu.dotenv.gradle</groupId>
<artifactId>co.uzzu.dotenv.gradle.gradle.plugin</artifactId>
<version>1.1.0</version>
<type>pom</type>
</dependency>
and
<dependency>
<groupId>co.uzzu.dotenv</groupId>
<artifactId>gradle</artifactId>
<version>1.0.1</version>
</dependency>
Which i got from:
`https://mvnrepository.com/artifact/co.uzzu.dotenv/gradle/1.1.0`
@maranza Even with the right repository configuration, you won't be able to use a Gradle plugin in Maven.
Hi @wilkinsona thank you for clarity. so what i am trying to achive is currently not possible. loading application.properties value from a .env file?
It's possible, but you'd have to write some code of your own to parse the .env
file and load it as configuration data. The io.github.cdimascio:dotenv-java
library that @x80486 linked to above may help you to do that.
If anyone's still interested in using
.env
as another properties file, for me it works by adding# Development properties (also used for docker-compose) spring.config.import=optional:file:.env[.properties]
into
application.properties
.The
[.properties]
is a hint for spring which translates to "Use this file without an extension as a.properties
file."
It would be great if this could work with spring.profiles.active
to fetch the correct .env, like:
spring.config.import=optional:classpath:.env-${spring.profiles.active}[.properties]
@membersound since spring.profiles.active
has a list of values, which .env
should spring load? imo this option is extremely difficult to implement for arguably little benefit
I forgot that multiple profiles can be added to that property (as I always only use one profile at once).
But well, in general the setup might be as follows:
application.properties
application-test.properties
application-production.properties
Now if each of the environments contains secrets that could be externalized into a .env
file, it would result in something like:
.env.example #listing all values that should be configured by the user. should not be loaded by spring, and is commited.
.env #general secrets that are valid for all profiles
.env-test #profile specific values
.env-production #profile specific values
How would you tell spring to load first the .env
, and then the profile-specific env?
Of course every application-*.properties
could set the spring.config.import
property itself, but I'd prefer having a one-line statement in only one file that covers all situations.
(because normally we have some kind of application-commons.properties
that all of our projects inherit from. And this commons project should define the spring.config.import
value for all .env
files, so that implementing projects don't have to care about .env configuration, but can simply throw them into the /src/main/resources
folder and it works).
A .env
file is usually meant to be used by you only when working in your desktop/laptop. On higher environments you just refer to the environment variables and act accordingly.
If so, where do you put secrets for test/production then (assuming a locally build app)?
At least they cannot be written into the application-*.properties
, as those are usually committed.
No, you refer to those secret values by using an environment variable in your application-*.properties
files:
# .env file
DATABASE_PASSWORD = "ThePassword"
SPRING_PROFILES_ACTIVE = "production"
# application.yaml
spring:
datasource:
password: "${DATABASE_PASSWORD}"
How do you fetch and provide those secrets is outside the scope of this issue. You can as well create a .env
file and source it with those values before starting the application — while using the same variables names.
If you have multiple .env[-profile]
files you will have the same problem as having those secrets in the usual .properties
files: where are you going to store them securely? On the other hand, if you have the means to store and retrieve secrets securely for each environment, but you can't provide them as environment variables, you could inject them by following the using the usual .env.template
file and replacing the values accordingly — resulting in just a .env
file in the end.
I would like to see the .env
file be used to setup development environment. These custom variables will be referred in my local development profile.
My typical usecase for .env
would be to have it on the project home directory.
File: .env
MY_KEY=My Local Value
...
On my local application.properties
or application.yml
file, would use it like,
my.app.properties=${MY_KEY} is working
. . .
Or
my:
app:
properties: '${MY_KEY} is working'
And expect my code using @Value("${my.app.properties}")
to be able to resolve it before doing DI.
I would like to see the
.env
file be used to setup development environment. These custom variables will be referred in my local development profile.
That would indeed be the ideal solution.
Any plans to implement suggested by @reflexdemon solution soon?
@petromir not really I’m afraid. We’re currently investing most of time on other issues. I suspect it might take a while for us to get to this one.
How to use env files with application.yml
config:
spring:
config:
import: optional:file:app.env[.properties]
datasource:
url: ${DATABASE_URL}
username: ${DATABASE_USER}
password: ${DATABASE_PASSWORD}
How to use env files with
application.yml
config:spring: config: import: optional:file:app.env[.properties] datasource: url: ${DATABASE_URL} username: ${DATABASE_USER} password: ${DATABASE_PASSWORD}
works fine!
I can confirm the spring.config.import: optional:
approach works fine both for local development and automated deployments, as follows:
With application.properties
:
spring.config.import=optional:classpath:.env[.properties]
spring.datasource.password=${DATABASE_PASSWORD}
Now you can create an .env
file only on your local development machine with:
DATABASE_PASSWORD=secret
As well, you can use your CI/CD deployment jobs to provide the secret from secret-store, and pass it to the spring application as environment variable, eg with docker-compose
:
services:
app:
environment:
- DATABASE_PASSWORD=$DATABASE_PASSWORD
Unfortunately, it was not possible o add the spring.config.import=optional:classpath:.env[.properties]
definition to a company-commons.properties
file in a company-commons.jar
library that is used by the implementation applications. The spring.config.import
setting must be repeated for any application explicit, and somehow cannot be inherited.