dotenv-vault icon indicating copy to clipboard operation
dotenv-vault copied to clipboard

Feature Request: Allow Partial Overrides for .env.[environment] Files Instead of Full Replacement

Open notnotjake opened this issue 1 year ago • 2 comments

Summary

Many modern tools—including Bun and dotenvx—have popularized the use of .env.* files (e.g., .env.production, .env.development) to override values in .env. However, dotenv-vault fully replaces the target environment when pushing a file, causing any variables not in the pushed file to be lost.

A partial override workflow would bring dotenv-vault more in line with the common “base + environment-specific override” pattern.

The Problem

  • Loss of Existing Variables

    • If .env.production doesn’t contain all necessary variables, pushing it to production removes any keys not listed.
  • Misalignment with .env.* Patterns

    • Tools like Bun, dotenvx, and others expect .env.production to override values from .env, not replace them entirely. Dotenv-vault's current behavior breaks this expectation.

Proposed Solution: Partial Override

  • Only override the keys present in .env.[environment].
    • If a key exists in .env.production, overwrite (or add) it.
    • If a key does not exist in .env.production, preserve its existing value in the vault.

How It Could Work

CLI-Level Merge

On dotenv-vault push production .env.production, the CLI would:

  1. Pull existing variables from production.
  2. Merge them with the local .env.production (where any duplicates in .env.production override existing values).
  3. Push the merged result back to production.

This ensures any variables not declared in .env.production remain unchanged in the vault.

Flags and Options

  • A --merge flag (or similar) to explicitly enable partial overrides.
  • A --all or --base option to push .env across all environments, providing a common starting point for overrides.

Limitations

  • Not Actually Syncing Layered Files

    • Pulling from the vault retrieves only the final compiled .env for a given environment (e.g., production, development). You don’t get separate .env and .env.production files back—just the merged result.
  • Potential for Environments Getting Out of Sync

    • Since dotenv-vault doesn’t natively support true layered environments, environments can drift apart over time.
    • Example scenario:
      1. You push .env (shared base variables).
      2. You push .env.production (overriding production-specific values).
      3. Later, you push .env again—this fully replaces the environment, wiping out values previously overridden by .env.production.
    • This means pushing .env later does not preserve per-environment overrides, requiring manual tracking of all expected values.
  • dotenv-vault design and use case needs

    • In a true layered system, you push .env once and then independently push per-environment overrides without losing values.
    • However, dotenv-vault only stores the final resolved environment, not the separate .env.* files.
    • Unlike Bun and dotenvx, where .env and .env.production are dynamically merged at runtime, dotenv-vault environments are static snapshots—meaning each push must fully account for all values, not just overrides.

Why This Proposal is Still a Good Compromise

  • Implementing true layered environments would likely require major changes to how dotenv-vault stores and manages variables.
  • This proposal minimizes destructive behavior by allowing partial overrides, while keeping the existing vault structure intact.
  • It still requires some manual tracking when updating environments, but it’s significantly less risky than the current full-replacement behavior.

Potential Impact

  • Infrastructure/Security

    • Minimal, as the logic change is entirely CLI-side. The vault would still store only the final merged result.
  • Backward Compatibility

    • Could be opt-in (e.g., via a --merge flag), allowing current full-replacement behavior to remain the default if desired.

notnotjake avatar Jan 10 '25 02:01 notnotjake

some great feedback here @notnotjake.

I'm the creator of both - dotenv-vault and dotenvx. right now i'm directing more of the team's efforts toward dotenvx and then longer term - beginning of 2026 i think - i'll begin merging both tools together. at that time we'll address all or most of this.

motdotla avatar Jan 10 '25 02:01 motdotla

@motdotla It would be great to have a feature where we can disable "synced" env environments on dotenv-vault. One where we can remove 1:1 syncing. I might want my .env.staging to stay completely different to my .env file. But if I push .env it looks like dotenv-vault is telling other environments to base off of .env or whatever the main is and I might want my .env.staging to be completely disconnected and not include some vars from .env or contain ones specific to staging and don't want that to trickle into other environments.

Hope I explained that well. Something that I'm running into and is somewhat similar to this feature request.

Which actually relates almost 1:1 of this mention in the main request: "Unlike Bun and dotenvx, where .env and .env.production are dynamically merged at runtime, dotenv-vault environments are static snapshots—meaning each push must fully account for all values, not just overrides." and or keeping various variables on specific environments and not all values on all environments.

Christian-Garrison avatar Feb 28 '25 14:02 Christian-Garrison