airbyte-platform icon indicating copy to clipboard operation
airbyte-platform copied to clipboard

🐛 Fix ENV VAR kv parsing to handle JSON values

Open bHacklv opened this issue 1 year ago • 5 comments
trafficstars

What

JOB_KUBE_ANNOTATIONS environment variable can be used to define one or more Job pod annotations. The valid value can be a JSON value (see example below). Current logic to process the key:value pairs is not allowing to pass the JSON value.

Before fix:

Image

Example of a valid istio k8s annotation that is incorrectly parsed without a fix:

proxy.istio.io/config={ 
"holdApplicationUntilProxyStarts": true, 
"proxyMetadata": { 
  "EXIT_ON_ZERO_ACTIVE_CONNECTIONS": "true" 
  } 
}

How

  1. Parse ENV VAR value with Regex Pattern that checks for JSON values
  2. Fallback to Regex Pattern that extracts k:v as it was done before with Splitter stream.

Recommended reading order

  1. airbyte-commons-with-dependencies/src/main/java/io/airbyte/commons/workers/config/EnvUtils.java
  2. airbyte-commons-with-dependencies/src/main/java/io/airbyte/commons/workers/config/WorkerConfigsProvider.java
  3. airbyte-commons-with-dependencies/src/test/java/io/airbyte/commons/workers/config/EnvUtilsTest.java

Can this PR be safely reverted and rolled back?

  • [x] YES 💚
  • [ ] NO ❌

bHacklv avatar Nov 06 '24 18:11 bHacklv