`meltano invoke` writes passwords into temporary config files during execution

[DouweM]

Meltano encourages you to manage passwords and other sensitive configuration using environment variables (or .env) so they don't end up checked into the repo, but does write these passwords down into the file system when generating the config files for Singer taps/targets, Airflow, or Superset (as of https://github.com/meltano/meltano/pull/6031). They'll also show up in meltano invoke --dump=config <plugin>, as that returns the exact file that's passed to the plugin in question. These files are cleaned up once meltano invoke completes (which is why --daemon flags don't work: https://github.com/meltano/meltano/issues/3440), but are still as long-living as your web UI or scheduler. This has potential security implications.

Since the superset_config.py config file we generate for Superset can contain arbitrary code, we could read the values from the environment variables there instead of hard-coding them into the file. This is not an option with Airflow or Singer, as airflow.cfg and config.json are simple serialized dictionaries.

[aaronsteers]

Logged here, follow up from our discussion on rewinding this for Airflow:

• https://github.com/meltano/meltano/issues/6110

It's different for Singer because we are the tool running the connector. But if we detect we are running an SDK tap or target (or another tap/target which supports --config=ENV), then even for those Singer plugins we could send all config via environment variables and completely avoid a stored file with cleartext secrets.

[stale[bot]]

This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.

[tayloramurphy]

Not clear if still relevant but would be good to investigate.