Ingestion framework logs plaintext passwords on error

[jasonstitt]

Affected module

Ingestion Framework

Describe the bug

Upon an error, the ingestion framework can log the contents of the configuration YAML file, which contains passwords in plaintext, such as the database password, and the bot JWT.

To Reproduce

Set up custom ingestion as documented. Run metadata ingest -c "config.yaml" with a config.yaml that contains an erroneous value.

Expected behavior

Any one of:

1. The config file can contain env var references instead of secrets
2. The config file is not logged
3. The config file is logged with redaction

Version:

• OS: linux
• Python version: 3.11
• OpenMetadata version: 1.3.2
• OpenMetadata Ingestion package version: openmetadata-ingestion = {extras = ["postgres"], version = "^1.3.2.0"}