fides icon indicating copy to clipboard operation
fides copied to clipboard

Allow per-client secrets (e.g. `FIDESCTL__CLI__ANALYTICS_ID`) to be configured separately from shared TOML config

Open NevilleS opened this issue 2 years ago • 1 comments

Is your feature request related to a specific problem?

Right now, on the fidesctl client, the fidesctl.toml file is used to configure:

  • organization-wide settings (e.g. shared fidesctl server host, port...)
  • user-specific settings (e.g. analytics opt-out)
  • user-specific secrets (e.g. analytics ID, and in the future we'll need to persist user auth as well)

When we mix these org-wide and user-specific concerns together in the same TOML file we're now running into issues where a version-controlled fidesctl TOML (e.g. https://github.com/ethyca/fidesdemo/blob/main/fidesctl.toml) starts conflicting with local changes. For example, a developer might:

  • Check out a project that is using fidesctl (e.g. https://github.com/ethyca/fidesdemo)
  • Start running fidesctl CLI commands, which generates an analytics_id and edits the fidesctl.toml file
  • Accidentally check-in the edited fidesctl.toml file with their user-specific secrets

Describe the solution you'd like

I'd like to see some way to separate the user-specific settings/secrets from the org-wide ones. This might mean having two separate files, some mixture of config & .env files, or some local storage. In other projects there is more separation of config & secrets: for example, see Meltano (https://docs.meltano.com/guide/configuration#configuration-layers) where config can be set in env variables (not shared, not persisted), a config file (shared, persistent), local database (not shared, persistent), etc.

Describe alternatives you've considered, if any

A few options might be:

  1. Never "auto-persist" the analytics_id ENV variable (or other user-specific secrets) when running in a container
  2. Store user-specific secrets in a separate file, like fidesctl.user.toml or .env.fides, etc.
  3. Store user-specific secrets in some local storage, like a sqlite db, a .credentials file, etc.

NevilleS avatar Apr 27 '22 15:04 NevilleS

I liked the way Meltano did this as well 👍🏽

Being able to have a clear definition for how to set env vars and use them would also give us a nice pattern to follow if we wanted to set the config values for scan and generate commands as well. We could certainly explore the same pattern with the system database as well for creating and storing config options either via the CLI or UI

SteveDMurphy avatar May 02 '22 15:05 SteveDMurphy