🚀 Feature Request: Allow to include secrets in wrangler's automatically generated type definitions

[Jackenmen]

Describe the solution

Problem

Currently, there is no way to let wrangler know about secrets so that it can include them in the automatically generated type definitions (wrangler types command).

Tried solutions

I can specify regular env vars in [vars] but when doing so with a secret, it becomes problematic later. For example, trying to set such var's value with wrangler secret put after deployment results in an error saying that the variable is already set and I have to delete all of those encrypted variables first.

I'm pretty sure this is not the intended way of doing things and prevents wrangler.toml from being the full source-of-truth which is its purpose (as documented). Cloudflare's Web UI even tells you to update the wrangler.toml file after updating the env vars with encrypted values, though it just suggests adding <empty textbox> (I had 4 secrets so it actually was a textbox with 4 newlines :smile:) to the wrangler.toml file.

Suggested solution

I think that the wrangler.toml should probably get a new secrets array specifically for specifying variables that are encrypted. Technically, it could make sense to allow specifying whether the variable should be encrypted directly in the [vars] table but this would likely be problematic as variable values are not limited to text values (see docs).

Additional context

I asked about this earlier this week on Cloudflare Developers Discord and was asked (by @ Skye-31) to make an issue: https://discord.com/channels/595317990191398933/799437470004412476/1234495956259704984

[DaniFoldi]

Hi @Jackenmen,

Happy to report that there was a recent change which allows wrangler types to pick up secrets as type string from your .dev.vars file 🎉

If you want to automate it, you can use wrangler secret list, transform that and save it as .dev.vars with any value, and then run type generation.

[Jackenmen]

Hmm, I didn't know that wrangler types picks up variables from .dev.vars.

Sadly, I don't think this solves my issue since the .dev.vars file is not something that should be put in the version control (it's even automatically added to .gitignore in wrangler-generated projects) and therefore doesn't serve as source-of-truth for the Worker configuration. If someone clones my repo and tries running wrangler types, the auto-generated Env interface would be different from what it is supposed to be simply because they didn't create a .dev.vars file with specific variables set (the opposite is also true if one includes some additional env vars in .dev.vars).

[yawnt]

👍🏻 same issue here

[aliyome]

You should put .dev.vars.example to your repository. I think that's enough

[Jackenmen]

That still requires that you create the .dev.vars file on clone (probably in CI or when a new user / contributor clones the repo), it might just make it slightly easier since you could presumably just cp .dev.vars.example .dev.vars to at least declare some (non-sense) values for the variables and therefore have wrangler types do the right thing. I feel like this is the type of thing that build tools should be abstracting for you so that you don't have to do anything special, if you're just doing typical things (and I don't think that having and using secrets in workers is something rare?). I think it is important for wrangler types to work on clone because the generated types are necessary for type checking to succeed (without the file, some Env attributes are going to be missing) and building shouldn't be requiring credentials (yes, they can be fake but why should they be needed in the first place?).

[Codex-]

I also encountered this cryptic behaviour, at first I was manually declaring an interface in addition to the generated one to give some typeness to the secrets being declared and then only by chance discovered that the typegen uses .dev.vars

I failed to find any documentation stating the inclusion of this file in the typegen.

It's a bit unintuitive, as the template / project generation builds a project that explicitly git ignores the .dev.vars file but then for type safety on the env it when generating the types .dev.vars is used?

• type-generation finding secrets (source)
• getVarsForDev hard coding .dev.vars (source)

I like the idea of being able to forward declare available secrets in wrangler, but I suppose since there is a disjoin between deploying secrets and wrangler.toml this may become unintuitive for other reasons.

Could we consider having a way to specify what files to read from for available secrets instead on the types command? This way we could simply target a .dev.vars.template or similar without committing the .dev.vars files to get consistent typegen. This also helps prevent developers from accidentally committing secrets

As a semi-related note, I also found it a bit difficult to understand from the documentation how secrets were treated. I understood it as "you declare the secrets as empty variables and the secrets are provided at runtime" but this isn't correct, and if you have a variable + secret of the same name, the variable takes precedence. Probably a me problem, would be nice to be explicitly clear about this behaviour in the documentation though

[kevinsimper]

@Codex- I would be a fan of defining them in wrangler.jsonc since it would be explicit and other programs that read wrangler.json would also be able to understand all keys to run the application

[TimStepanovAtBrandBoosting]

Could we perhaps get an update on this? This would be a huge QOL improvement.

Our current workaround is to just include the .dev.vars file in the git repository with dummy values for the secrets. This is ugly and prevents using them for local development.

[LeonBlade]

Cloudflare's intention behind environment variables is that you should be using the .dev.vars for your local environment variables including secrets. With that in mind, you still need to communicate your secrets across the dev team either way, afterwards running the typegen will generate the proper env variables.

That being said, I'm not sure if there is any downside to having typegen generate these except for in niche situations where you're using local env vars that aren't deployed as secrets in the project yet. However, this alone may be reason not to support something like this.

My understanding of this may be misguided however, so please feel free to correct me if you think I'm wrong. I do think providing a .dev.vars.example file or something equivalent is a good start and you can communicate on how to adjust this for development with the team.

[jared-figma]

We're running into this too and have also landed on a solution of checking in a dummy .dev.vars file with empty strings for secrets, which isn't ideal

Could a solution be to just declare secret names in wrangler.toml somehow and have type-generation use that? Then the way that the secret is actually provided (.dev.vars locally, the secrets primitive in deployed workers) could be decoupled from the Wrangler declaration and the resulting types.

[comatory]

I find the approach taken to generating types non-transparent, and it's causing me (and probably some other folks) to figure out the work-arounds on my own. I would assume wrangler would choose a golden path, and use defaults that are in-line with the most common scenarios. One of these scenarios has to be Typescript apps that use secrets, given Typescript/Javascript is the de-facto language of workerd runtime.

Checking-in .dev.vars to version control is a non-starter for us, as developers should be encouraged to fill in their own secrets as they work on the app. This is especially true in professional environment, with multiple people working on the same app. It would be very easy for a developer to add their secret there and mistakenly commit the file to version control, .dev.vars should be in .gitignore to avoid this specific scenario.

Yes, I could create some kind of tooling, that would generate .dev.vars for everyone collaborating on the project, but it's additional code that needs to be maintained.

I am proposing an idea which would be more logical, and would allow local development and correctly generated types at the same time.

1. Introduce .dev.secrets file just for storing secrets. .dev.vars assumes it's only for variables - confusing. Both of these files should not be checked into version control and should only be used for local development.
2. Introduce new secrets field to wrangler.jsonc/wrangler.toml. It would have same topology as vars field. Example:

   {
      "secrets": {
            "MY_SECRET": "{{ wrangler }}"
      },
      "production": {
          "secrets": {
              "MY_SECRET": "{{ wrangler }}"
          } 
   }
   

   The {{ wrangler }} would be just a placeholder, denoting that this value gets replaced during runtime. It's debatable, whether any other value should be allowed.
3. The command wrangler types would then ignore .dev.vars and .dev.secrets completely, because it's non-transparent and not everyone knows it's using those files. Wrangler configuration file should be the source of truth anyway. These .dev.* files would only be used during local development to replace / override existing values in the wrangler configuration.

This is just an idea, but I think it would make the whole thing more straightforward, instead of stranger workarounds[^1].

[^1]: In our case, we run wrangler types command in CI. We don't store the generated types in version control, because they're huge and they are basically build artifacts. So our only solution was to expand the Env binding to include our secrets, e.g. Env & { MY_SECRET: string }. This works but it's not very ergonomic, as you can forget to add/remove values from the type definition.

[nickdandakis]

Ran into the same issue and also didn't want to check in a .dev.vars.

Wrote this one-liner instead that I run alongside npx wrangler types

npx wrangler secret list | grep -o '"name"[[:space:]]*:[[:space:]]*"[^"]*"' | cut -d'"' -f4 | awk 'BEGIN{print "interface Env {"} {print "  "$0": string;"} END{print "}"}' > worker/Env.d.ts

Pulls secret list and writes into a type definition file (replace worker/Env.d.ts to wherever you want the type definition file to land

[Jackenmen]

It seems that the wrangler types command has recently (#9914) received a new flag - --env-file - which can be used to specify an additional env file that the env vars can be taken from. This at least makes it possible to list secrets in a .dev.vars.example file, check it in (since it's an example file) and point wrangler at it: https://github.com/Jackenmen/cf-email-worker-transcribe-audio-attachments/commit/8c2e3ef6c0eee28ca1666224c0c137b1fbb5f61c

I still consider it a workaround, but at least it doesn't require writing additional scripts.

[chris-martin]

It seems that the wrangler types command has recently (#9914) received a new flag - --env-file - which can be used to specify an additional env file that the env vars can be taken from.

Thank you for that suggestion.

I still consider it a workaround, but at least it doesn't require writing additional scripts.

If the intro tutorials were thoroughly updated and folks were writing package.json scripts that way using --env-file from the get-go, I don't think it would feel like a workaround. IMO at this point this could be considered a documentation issue only.

[Hebilicious]

I like the suggestion of adding secrets to wrangler.jsonc. Any of these syntax would be fine imo.

	"secrets": {
		"FOO": {},
		"BAR": {}
	},
	"secrets": [
		{ "name": "FOO"},
		{ "name": "BAR"}
	],
	"secrets": ["FOO", "BAR"]

I prefer the first once since it would be more futureproof with the possibility of passing options to each secrets.

[helle253]

I've been wrestling with this issue for a few days now - it seems really weird for the generated type file to be the 'source of truth' for worker configurations, but have functionally no way to tell it about the existence of secrets...

We have secrets_store_secrets which sort of works this way, but also has some additional overhead (afaik those aren't usable in local development contexts)

It strikes me as super weird to use another file (e.g. .dev.vars or another .env file for the 'source of truth'.

defining a set of secrets in the wrangler config file, whose values are populated asynchronously from the type-gen step, seems like the straightforward approach. It's unclear to me why there are all these hoops to jump through for that relatively basic functionality.

[Hiutaky]

I've developed a workaround script to generate types for our Cloudflare secrets, ensuring type safety in our local development environment.

This script automates the process by parsing the output of wrangler secret list, temporarily adding the secret keys to wrangler.jsonc, running the type generator to create cloudflare-env.d.ts, and then restoring the original configuration.

This solution is designed for local development only the content of cloudflare-env.d.ts and restore the real wrangler.jsonc content so it's production ready again.

I then updated the command for cf-typegen:

{
 "scripts": {
     "cf-typegen": "bun scripts/createWranglerEnv.ts"
 }
}

// file : ./scripts/createWranglerEnv.ts
import Bun from "bun";
import * as parser from "jsonc-parser";

const CONF = './wrangler.jsonc';

async function main() {
    const listopt = Bun.spawnSync({
        cmd: ['bunx', 'wrangler', 'secret', 'list']
    });

    const list = (JSON.parse( listopt.stdout.toString() ) as Array<{
        name: string;
        type: string;
    }>).map( e => e.name );

    const wranglerConfJsonc = await Bun.file(
        CONF
    ).text();

    try {
        const wranglerConf = parser.parse(wranglerConfJsonc)
        const vars = wranglerConf.vars;
    
        const keys = Object.keys(vars);
    
        for( const secret of list )
            if( ! keys.includes(secret) )
                wranglerConf.vars[secret] = "{{REDACTED}}";
    
        await write( JSON.stringify( wranglerConf ) );
        const cftypegen = Bun.spawnSync({
            cmd: ['bun', "wrangler", "types", "--env-interface", "CloudflareEnv", "./cloudflare-env.d.ts"]
        });
        if( ! cftypegen.exitCode  ) {
            await write( wranglerConfJsonc )
        }
    } catch (e ) {
        await write( wranglerConfJsonc )
        throw e;
    }
}

async function write( buffer : any) {
    await Bun.write(CONF, buffer);
}

main().catch( e => console.error)