airbyte icon indicating copy to clipboard operation
airbyte copied to clipboard

bulk-cdk: add feature flag environments

Open postamar opened this issue 1 year ago • 6 comments

What

Fixes https://github.com/airbytehq/airbyte-internal-issues/issues/9704

The Bulk CDK needs to support the DEPLOYMENT_MODE environment variable and the associated CLOUD feature flag. This PR adds support for this and makes source-mysql-v2 adopt it.

How

As far as I can tell, none of the other feature flags from the legacy CDK matter for connectors. In any case, the Bulk CDK now has a Micronaut Factory for injecting Set<FeatureFlag> where FeatureFlag is an enum and the injected value is the set of active feature flags. This is used in source-mysql-v2 to cause CHECK to fail when a config doesn't have enough encryption.

Review guide

Commit by commit

User Impact

None

Can this PR be safely reverted and rolled back?

  • [x] YES 💚
  • [ ] NO ❌

postamar avatar Oct 09 '24 21:10 postamar

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 18, 2024 2:25am

vercel[bot] avatar Oct 09 '24 21:10 vercel[bot]

@johnny-schmidt you already did something for this I think? at least destination-dev-null respects cloud vs oss mode

edgao avatar Oct 09 '24 21:10 edgao

Interesting! I searched and found it in the test fixtures, the substring to search for is "deployment.mode". In my PR I hadn't thought of how this would impact the test runners, I'll have to figure something out. Back to the drawing board!

postamar avatar Oct 10 '24 11:10 postamar

This is ready for review, the only failing checks are connnector version bumps.

postamar avatar Oct 10 '24 16:10 postamar

@johnny-schmidt you already did something for this I think? at least destination-dev-null respects cloud vs oss mode @postamar @edgao

Sorry, I missed this earlier. Micronaut automagically propagates env variables as properties. You get "deployment.mode" and any other env variable for free. The tests override the properties so that we can test both envs, but specifying them as properties isn't necessary in production.

I'm not sure what this PR accomplishes on top of that other than to limit the list of supported vars? However, you can also get that by specifying includes/excludes when building the ApplicationContext.

johnny-schmidt avatar Oct 14 '24 16:10 johnny-schmidt

https://docs.micronaut.io/latest/api/io/micronaut/context/ApplicationContextBuilder.html#environmentVariableIncludes(java.lang.String...)

johnny-schmidt avatar Oct 14 '24 16:10 johnny-schmidt

I'm not sure what this PR accomplishes on top of that other than to limit the list of supported vars?

Limiting the supported vars is not the goal per se but there is value in having a clear catalog of which env vars are meaningful. The main purpose of this PR is to facilitate activating feature flags in regular junit tests which are not @MicronautTests. I understand that this may not be relevant to destinations.

postamar avatar Oct 15 '24 13:10 postamar

No worries! Like I said it's not urgent and I only just got the build to pass. Thanks for taking a look.

postamar avatar Oct 17 '24 23:10 postamar