Allow Multiple Dotenv Files

[mikeyhew]

This adds support for multiple dotenv files by updating the dotenv-filename and dotenv-path settings to support an array of strings instead of just a single string. Files are loaded in the order specified in the array, and variables in later files override variables in earlier files.

Some edge cases to point out:

• with dotenv-filename, if any of the files in the list is found in the current working directory it will stop there and not move up to a parent directory
• if an empty array is supplied for dotenv-path then that is the same as not supplying dotenv-path, and it will fall back to dotenv-filename
• if an empty array is supplied for dotenv-filename then it will not default to ".env", it just won't load from a file at all.

We could add some checks for some of these edge cases if they are too odd and disallow them with an error. But I get the sense from looking at the code and tests surrounding dotenv loading that the philosophy in Just is to not worry too much about disallowing weird edge cases.

Note that there is one potential breaking change here: this updates the dotenv_filename and dotenv_path settings to be Vec<String> and Vec<PathBuf> instead of Option<String> and Option<PathBuf>, so when you call just --dump --dump-format=json, they will be output as arrays instead of as a single string or null. If we need to guarantee backward compatibility for json dumps then it's possible to change this to output as null or a single string if there are less than 2 entries in the array, at the expense of making the output more complicated to parse.

See this comment for more details on the design for this PR.

Fixes #1748

[mikeyhew]

I understand @casey's pretty busy right now so if anyone else wants to take a look at this and give me any feedback, feel free. In the meantime I've been test driving this locally and it's been working well

[mikeyhew]

@casey just pinging you because I'd love to get this feature in, and also if you could do whatever you need to do to get CI to run that'd be helpful because I could try and fix any errors that come up

[kwuite]

+1 for this feature PR pretty please

[gsemet]

That would be very useful.

here my use case:

• i have credentials and other multi-project environment variables stored in a parent folder .env (~/Projects/.env). This typically my username, some creds, things that we can have in CI.
• in a given project, i may have some configuration step that will configure some settings, so i may write .env.local. This file is intended into being commited in conf.
• user may override some settings through .env.user, allowing tests and so on.

in sort, that would require to merge based on the environment var name, with the left-most precedence.

[mikeyhew]

@gsemet usually I've seen .env.local be the file that's not committed (the user's local overrides). But that said you can name the files however you want in your project. As implemented though I don't think you'd be able to do things quite the way you described because it stops at the first directory that contains any of the files in the dotenv-filename array. You'd have to have all your dotenv files in the same directory

[mikeyhew]

@casey I'd love to get this in soon, it would enable the very common workflow of one .env file that gets committed and a .env.local file for local overrides. Any chance you can take a look at this soon? Or if not, can you at least enable CI so I can fix any errors that come up?

[mikeyhew]

Looks like there's some conflicts from the dotenv-override setting that got merged, I'll try and fix them today

[mikeyhew]

@casey can you approve the workflow run, and ping me when you do?

[kwuite]

@casey @mikeyhew , is there anything I can do to support the release of this feature?

[mikeyhew]

@kwuite are you a maintainer and can you approve the workflow run? Otherwise you could install this branch locally to try it out and let me know if you run into any issues:

cargo install just --git https://github.com/mikeyhew/just --branch multiple-dotenv-files

[kwuite]

@mikeyhew , my apologies, I am not a maintainer of this project. I am currently developing a DevOps CLI interface for my startup, ASD Engineering, and my tool is designed to work like a git submodule.

I do not want my .asd code to get mixed up with project specific environment variables. The environment variables are consumed from multiple folders, with module-specific .env files stored in a workspace. However, all these values now need to be merged into the user repository .env file, because I’m not allowed to import them from two separate locations (the user’s repo and my .asd/workspace/.env file).

I wrote some decently hacky code to get this working but I would love to get rid of it and use a pure just approach.

A few weeks back I found your PR and was hoping for a merge but the way it looks this code will not get merged any time soon.

I would be happy to review your code, test it, validate it against my own (currently private .asd) implementation, and if everything works well, build and publish a binary via GitHub similar to the original just creator. Would that help you in any way? Allthough I am still calculating myself whether this is the route I want to take🤔.

[mikeyhew]

@kwuite I'd love it if you'd be willing to test it out, provide feedback etc. Just keep in mind there's still no guarantee this gets merged in its current state, I'm not sure what @casey's stance on it is since I haven't heard from him yet. It looks like he did enable the CI run though so that's a good sign.

[kwuite]

@mikeyhew I have schedule 2 hours for testing tomorrow afternoon. Will provide feedback in this channel. Have a great day👍

[groutoutlook]

I personally think this use case is only applicable in very exotic use case though... IMHO just should not involve too much on fetching a global data source as filesystem there, and instead trying to keep its state between recipes/inside justfiles. There are already direnv which can deal with this relatively well isn't it? Maybe we could try to explore the direction as, making just talk to those kinds of applications easier.

[jceb]

@groutoutlook I like the idea of making just play nicely with direnv etc., however these tools come with their own challenges. direnv requires a prior direnv allow call, which in turn provides the ability to execute arbitrary code.

I prefer a native solution in just to safely load multiple environment files before exploring the integrating with other tools.