ConduitIO
Feature: Include what fields are being deprecated by the connector
Feature description
Over time, connectors may deprecate configuration fields in favor of others. While these changes are documented in the readme (see conduit-connector-postgres and conduit-connector-kafka), we need a programmatic way to determine which fields are deprecated so clients can hide them from their configuration.
~Update: Consider a way to mark fields as sensitive if we consider it appropriate as well. This is for clients to determine in which situations is really important to obfuscate its values. If scope becomes to big, consider creating a separate issue for this.~ ⬅︎ This issue other was created for this purpose.
This includes an addition in the connector protocol, to indicate if a specific parameter is deprecated (boolean field), a change in the SDK and ideally also support to populate the new field using paramgen. The deprecation can be detected if a field comment starts with Deprecated: (same as the idiomatic Go way of deprecating fields).
I'm not sure if this belongs here or would be better handled in a separate issue.
But we have other config tagging needs outside of just what is deprecated. We also need a way to indicate if a field is sensitive or contains sensitive information so that we might automatically know when to obfuscate values on the platform.
Adding this consideration here in case there's some potential design pattern that can be derived from the requirements.
Update after today's Conduit weekly. I'll spike on this to get a sense of the scope we're dealing with. Hope to come back with better info around this time tomorrow about whether we'll be implementing this now or later.
Created a discussion item here https://github.com/ConduitIO/conduit/discussions/1611.
Current estimate for implementation is (at least) two days. Moving back to To do again until we start implementation.