Support for .env files

[thebarndog]

Problem

Modern applications deal with all kinds of environment variables, from build numbers to api keys that can be injected on CI or with tuist, via the command line. Unfortunately however, tuist doesn't support putting those environment variables in a .env file that can be read in. There's a couple of reasons that tuist should consider adding support for declaring environment variables in this way:

1. tuist is great because it's a standard command; I can run tuist build or tuist generate without worrying about anything else but once you start adding in environment variables, the command becomes VAR=1 VAR2=2 tuist generate and so on and becomes more cumbersome and at that point, I just end up writing a script to run the tuist command that passes in environment variables.

2. Keeping environment variables in a .env file would align more with standard practice around handling of environment variables and enable developers to keep those values locally, not in version control, without the need for intermediary scripts. It would become far easier to store things in the env file such as build numbers for the application, rather than having to remember it.

I actually ended up writing my own library that I'm using in a script:

import Darwin
import ShellOut // [email protected]:JohnSundell/ShellOut.git == 2.3.0
import SwiftDotenv // [email protected]:thebarndog/swift-dotenv.git == 1.1.0

let environment = try Dotenv.load()

guard
    let bundleID = environment["BUNDLE_ID"],
    let teamID = environment["TEAM_ID"],
    let buildNumber = environment["BUILD_NUMBER"] else {
        exit(EXIT_FAILURE)
    }

let output = try shellOut(to: "TUIST_BUNDLE_ID=\(bundleID.stringValue) TUIST_DEVELOPMENT_TEAM=\(teamID.stringValue) TUIST_BUILD_NUMBER=\(buildNumber.stringValue) tuist generate")
print(output)

[danieleformichelli]

Hello @thebarndog 👋 Could you please better explain your use case (e.g. why do you need env variables, where do you store .env files, and so on)?

There is a PR on a similar topic and we would like to better understand the use case before deciding how to proceed 🙏

[thebarndog]

Sure thing! It's basically the same use case as on the server or a ruby application; you use .env files to store sensitive credentials such as API keys, secrets, etc, things that you don't want committed to version history. The .env file is gitignored and only exists locally so these secrets never end up in git. Additionally you can have env files for different environments like staging, production, and development, to easy switch between different keys, credentials, etc for services.

In my use case, I'm storing things like my Apple Team ID and my application's build number. Because I'm using tuist, I have to pass a build number into the project manifest and every time I want to ship a TestFlight build, I have to manually increment that number. To get around that, I have the build number in a .env file and a script that increments that value via my SwiftDotenv package.

It would be really great if the Tuist Environment object could pick up on my environment file so that I can invoke tuist generate without having to prepend environment variables to the command.

[danieleformichelli]

So in your case you would have multiple env files, as some are committed (version file) and some not (secrets file), right?

[ezraberch]

How about using project description helpers to solve this problem instead? Define a Secrets struct inside the version-controlled part of your code then instantiate it (or multiple versions of it) in .gitignored files.

[danieleformichelli]

I agree with @ezraberch 👍 It would be slightly harder to change values via scripts, but still doable 🚀

[kwridan]

Another suggestion is creating a small wrapper script that sources the desired env files:

.env:

export TUIST_VAR_A=abc
export TUIST_VAR_B=def
# ....

generate.sh:

source .env
tuist generate

[thebarndog]

I agree with @ezraberch 👍 It would be slightly harder to change values via scripts, but still doable 🚀

Because that's just not standard. If any part of your application isn't in Swift, like fastlane for example, then it can't use that struct. If there was support for .env across the board, then all parts of the application, regardless of the language, can read them in.

So in your case you would have multiple env files, as some are committed (version file) and some not (secrets file), right?

No, none of them are committed to version control.

[danieleformichelli]

Can you make a concrete example of something needed by both Project.swift and fast lane for example?

[thebarndog]

Can you make a concrete example of something needed by both Project.swift and fast lane for example?

I did, the build number and Apple team id. Fastlane needs to increment the build number when I ship a build, which also has to be passed into tuist so that the project gets generated with the build number correct. And both the project manifest and fast lane need the apple team id.

[thebarndog]

@danyf90 I dont think I explained the use case well enough so let me try again:

When you run a fastlane task, let's say "ship to TestFlight", fastlane has to increment your build number via agvtool and that build number, in non-tuist projects, sits in the project file as well as in the info plist. Because the entire point of tuist is to get rid of the project file, that build number has to go somewhere. If its only in the Project.swift manifest file, then fastlane can't easily access that number and increment it. But if the build number is shared via a .env file, which ruby has fantastic support for, fastlane can change that number and tuist, in theory, can also pick up on it when the project file gets generated.

This of course is a somewhat trivial example of why they'd be useful; the best use case would probably be for loading secret keys and whatnot into the app. Obviously you don't want your API keys to be committed to version control so putting them in a .env file that exists only locally, that tuist can then read in during project generation, time is ideal.

[danieleformichelli]

When you run a fastlane task, let's say "ship to TestFlight", fastlane has to increment your build number via agvtool and that build number, in non-tuist projects, sits in the project file as well as in the info plist

I'm not familiar with how this flow works, but I guess fastlane builds are done in a script or in CI, in which case you can just do

source .env
tuist generate
…

in the script or ci configuration, isn't it?

the best use case would probably be for loading secret keys and whatnot into the app

For this scenario, you can still put the secrets in a ProjectDescriptionHelper (not committed to git), isn't it?

[ladislas]

then fastlane can't easily access that number and increment it

You can have fastlane pull the current build number on the AppStoreConnect and increment this one.

That's what we do, we have something like this:

def get_latest_build()
	# Login to ITC to get information about the current version
	Spaceship::ConnectAPI.login(@username)
	app = Spaceship::Tunes::Application.find("<bundle_id>")

	# Check if current version exists in ITC
	version_number = get_local_version
	train = app.build_trains["#{version_number}"]

	if train == nil
		puts "Version #{version_number} is not yet present on ITC. Setting build_number to 1"
		return 1
	else
		latest_build_number = latest_testflight_build_number(version: "#{version_number}")
		new_build = latest_build_number.to_i + 1

		puts "Setting new build number to #{new_build}"
		return new_build
	end
end

It was written a long time ago, still works but maybe fastlane now provides native solutions.

See https://docs.fastlane.tools/actions/latest_testflight_build_number/

[thebarndog]

I'm not familiar with how this flow works, but I guess fastlane builds are done in a script or in CI, in which case you can just do

source .env
tuist generate
…

in the script or ci configuration, isn't it?

You can but in my case, where I share env variables among multiple scripts, it doesn't work because Tuist requires variables in the form of TUIST_ENV_VAR prefixed with TUIST, which is an entirely different issue. And using source sources it into the entire environment rather than tuist loading a specific subset just from that file, which is entirely the point. I don't want to source my env vars, I want tuist to take an argument that points to a .env file for my specific environment variables and nothing else.

the best use case would probably be for loading secret keys and whatnot into the app

For this scenario, you can still put the secrets in a ProjectDescriptionHelper (not committed to git), isn't it?

You could but that's just not standard. env files have been around for a long long time and they're the standard for loading in environment variables on servers and scripting alike. Just because they can be put in a ProjectDescriptionHelper does not mean they should be put there. We shouldn't be inventing new standardizations or ways to do things just so they fit better in tuist; we should adapt to the pre-existing standard which again is .env files.

[thebarndog]

then fastlane can't easily access that number and increment it

You can have fastlane pull the current build number on the AppStoreConnect and increment this one.

That's what we do, we have something like this:

def get_latest_build()
	# Login to ITC to get information about the current version
	Spaceship::ConnectAPI.login(@username)
	app = Spaceship::Tunes::Application.find("<bundle_id>")

	# Check if current version exists in ITC
	version_number = get_local_version
	train = app.build_trains["#{version_number}"]

	if train == nil
		puts "Version #{version_number} is not yet present on ITC. Setting build_number to 1"
		return 1
	else
		latest_build_number = latest_testflight_build_number(version: "#{version_number}")
		new_build = latest_build_number.to_i + 1

		puts "Setting new build number to #{new_build}"
		return new_build
	end
end

It was written a long time ago, still works but maybe fastlane now provides native solutions.

See https://docs.fastlane.tools/actions/latest_testflight_build_number/

@ladislas sure you could but why introduce unnecessary network calls into your script when it can be as simple as reading in a file, incrementing a value, and saving to that file.

[thebarndog]

How about using project description helpers to solve this problem instead? Define a Secrets struct inside the version-controlled part of your code then instantiate it (or multiple versions of it) in .gitignored files.

@ezraberch sorry forgot to respond to this. It's because it's not standard. No one outside of the Tuist ecosystem knows what tuist is but everyone, including non iOS developers, knows what .env files are. Doing what you suggest just makes it way harder to onboard new people because thats such a tuist specific thing but if it's in the .env file, everyone knows where they exist and how to modify them. For example, a server side engineer could come into my project and tweak the environment variables on their own without any outside input but if that stuff is defined in Tuist, then they'll immediately require outside help and context on how to change stuff.

The TL;DR to this is tuist should support .env files because they're widely used, easy to modify, and not tuist specific. Im all for using tuist and I love the tool for automating my iOS setup but tuist should not be in the business of redefining environment variable paradigms just for the sake of keeping everything in tuist, it's bad practice.

[hisaac]

I know I don't hold all that much sway here, but for what it's worth, I agree with @thebarndog on this. .env support would be nice to have in Tuist.

[thebarndog]

Don't want this issue to become stale so I figured I would come back and comment on it. It seems that this feature would be useful, at least for those that want to use it. I think it's important too that tuist not be prescriptive and that it allows for flexibility. As I mentioned previously, .env files are standard across many languages and software disciplines and many people are familiar with them. Supporting them would ease integration pain for those that aren't familiar with tuist but are familiar with .env files. It's also worth noting that tuist is very specifically an iOS tool and keeping the environment variables solely within that framework precludes other non-iOS/Swift developers from contributing. A ruby engineer could for example, tweak values in a .env file without needing to know how tuist works.

@danyf90 maybe it's worth resurrecting #4125? I'm not sure what it involves really as I'm not too familiar with the workings of tuist but I'd be glad to help where I can.

[danieleformichelli]

I think it's reasonable to support it.

Before doing it let's see what the others @tuist/core think about it 🙏

[luispadron]

Wasn't this added before in https://github.com/tuist/tuist/pull/4125 and closed for caching issues?

Ah nvm, I see we're discussing it here. I'm not opposed to it if people find it useful.

[thebarndog]

Appreciate ya'll :)

[danieleformichelli]

Closing this as it seems there is no longer much interest, @thebarndog @jaerod95 feel free to open a discussion to decide how to proceed if you are still interested in it

[denizdogan]

For what it's worth, I was looking for this feature just now, so I'm interested in having this feature.

[thebarndog]

I managed to finally get some time and put up a WIP PR. It's mostly finalized, I'm just debating on whether or not to add tests to Tuist since the underlying package, swift-dotenv is already tested and adding tests in Tuist would be kind of redundant.

Please take a look and give your feedback! Glad to discuss any changes, will try to get to them as quickly as I can.