[TRI-1411] feat: Add support for Preview environments

[matt-aitken]

We currently support three environment types

1. Dev (your local machine, we create one of these for each team member of a Project)
2. Staging
3. Production

You can't create more than just these environments, and staging and production can only expose a single endpoint URL.

Problem

Vercel, and other hosts, generate unique sites for things like Pull Requests. We don't currently support these.

Solution

Add support for Preview environments to be created. This should probably be via the API/SDK. But also having an integration with Vercel that automatically does this would be good.

_{From SyncLinear.com | TRI-1411}

[hmacr]

@matt-aitken One way to solve this is, when the Trigger client is initialized we can check the deployment type (Vercel & Netlify expose it as an env-var). For a preview deployment, an API (eg. create-runtime-environment) in the trigger.dev server will be invoked which will return a new pair of API keys for the preview environment. The client initialization will be completed with the returned keys. If the API fails, the client falls back to the keys from the env-vars of the deployment.

create-runtime-environment API needs to be authorized with a new key (eg. project-key) since it doesn’t fall under the scope of environment-specific actions, so we can't use the environment API keys.

Your thoughts?

[ericallam]

@hmacr I think the preview environment support needs to be developed while creating our vercel integration (which gives us the webhooks we'd need to create/teardown preview envs). We should really explore what a Vercel integration would take as part of this issue

[jacobparis]

This would also be needed for Cloud Dev Environments like Gitpod or Github Codespaces, which run on a unique URL for each workspace

[ericallam]

@jacobparis yea sorry we've got a rogue bot 🙅

[hmacr]

The way I understand this use-case is that Vercel (or any deployment host) would create a preview environment and we want to create unique set of API keys for each of those environments.

The flow would be: Create PR/Add Commit to an existing PR → Vercel initiates a preview deployment → "trigger.dev" server listens to the deployment.succeeded webhook → Create a new set of API keys → Let the preview environment use these keys for the API calls.

The problem then becomes, how do we inject the new keys as env-vars and make the service use it? Vercel has an API (create-env-var) to inject env-vars to a specific preview deployment, and an API (create-deployment) to re-deploy an existing deployment which will essentially restart the server.

Do we really want to restart the server just for the env-vars to be picked up? Luckily, this is only needed for the initial preview-env creation, as the subsequent preview-env re-deployment (as a side effect of updating a PR) would already have the env-vars in place.

On the flip side, Vercel does not delete the preview environment when the associated branch is deleted. The user has to delete the environment by using the dashboard, the CLI, or the API (re: Vercel #8165). Also, no webhooks are sent when an environment is deleted (this might be a blocker to use Vercel integration). Ref: Deployment Webhooks Sent by Vercel

@ericallam Did you have a different proposal for the flow when this feature was ideated?

[sethtjf]

Jumping into this conversation because I was searching for preview support when I integrated trigger yesterday. Couldn't Trigger add a "preview" environment and issue one set of credentials for it that all preview environment branches use? It seemed that the complexity is attributable to figuring out how to inject unique environment variables for each preview environment, but I don't see why that's necessary. The Trigger webhook url could be generated for each preview branch from the deployment url. Are unique credentials required?

[martijnarts]

Is it correct that currently, using Trigger.dev means you cannot use Trigger jobs from Vercel Preview environments at all?