airbyte icon indicating copy to clipboard operation
airbyte copied to clipboard

Published 20 hours ago • verified publisher icon airbytehq

[deploy] Worker failing to start with external S3 configuration

Open damian-aifinyo opened this issue 1 year ago • 4 comments

Helm Chart Version

0.64.185

What step the error happened?

On deploy

Revelant information

I'm trying to deploy airbyte on my EKS cluster (I followed indications from https://docs.airbyte.com/deploying-airbyte/on-kubernetes-via-helm), I configure and external S3 bucket and an external DB (hosted on RDS). Everything seems to be correct during deploy I can even access the web panel. But the problem is the worker, it never finish to start, the error mention that the region should not be empty. I configure it as indicated. I don't know if I need to define it somewhere else. Here is my values file and the extra resources (secrets)

global:
  database:
    secretName: "db-secrets"
    secretValue: "DATABASE_PASSWORD"

  state:
    storage:
      type: "S3"

  logs:
    accessKey:
      password: ""
      existingSecret: "airbyte-logs-secrets"
      existingSecretKey: "AWS_KEY"
    secretKey:
      password: ""
      existingSecret: "airbyte-logs-secrets"
      existingSecretKey: "AWS_SECRET_KEY"

  storage:
    type: "S3"

  minio:
    enabled: false
    nodeSelector: {}
    tolerations: []
    affinity: {}

worker:
  extraEnv:
    - name: AWS_ACCESS_KEY_ID
      valueFrom:
        secretKeyRef:
          name: airbyte-logs-secrets
          key: AWS_KEY
    - name: AWS_SECRET_ACCESS_KEY
      valueFrom:
        secretKeyRef:
          name: airbyte-logs-secrets
          key: AWS_SECRET_KEY
    - name: STATE_STORAGE_S3_ACCESS_KEY
      valueFrom:
        secretKeyRef:
          name: airbyte-logs-secrets
          key: AWS_KEY
    - name: STATE_STORAGE_S3_SECRET_ACCESS_KEY
      valueFrom:
        secretKeyRef:
          name: airbyte-logs-secrets
          key: AWS_SECRET_KEY
    - name: STATE_STORAGE_S3_BUCKET_NAME
      valueFrom:
        secretKeyRef:
          name: airbyte-logs-secrets
          key: S3_LOG_BUCKET
    - name: STATE_STORAGE_S3_REGION
      valueFrom:
        secretKeyRef:
          name: airbyte-logs-secrets
          key: S3_LOG_BUCKET_REGION

server:
  extraEnv:
    - name: AWS_ACCESS_KEY_ID
      valueFrom:
        secretKeyRef:
          name: airbyte-logs-secrets
          key: AWS_KEY
    - name: AWS_SECRET_ACCESS_KEY
      valueFrom:
        secretKeyRef:
          name: airbyte-logs-secrets
          key: AWS_SECRET_KEY
    - name: STATE_STORAGE_S3_ACCESS_KEY
      valueFrom:
        secretKeyRef:
          name: airbyte-logs-secrets
          key: AWS_KEY
    - name: STATE_STORAGE_S3_SECRET_ACCESS_KEY
      valueFrom:
        secretKeyRef:
          name: airbyte-logs-secrets
          key: AWS_SECRET_KEY
    - name: STATE_STORAGE_S3_BUCKET_NAME
      valueFrom:
        secretKeyRef:
          name: airbyte-logs-secrets
          key: S3_LOG_BUCKET
    - name: STATE_STORAGE_S3_REGION
      valueFrom:
        secretKeyRef:
          name: airbyte-logs-secrets
          key: S3_LOG_BUCKET_REGION

postgresql:
  enabled: false

externalDatabase:
  host: <my-host>
  user: <my-user>
  database: airbyte
  password: ""
  existingSecret: "db-secrets"
  existingSecretPasswordKey: "DATABASE_PASSWORD"
  port: 5432
  jdbcUrl: ""

apiVersion: v1
kind: Secret
metadata:
 name: airbyte-logs-secrets
type: Opaque
stringData:
 AWS_KEY: <my-key>
 AWS_SECRET_KEY: <my-secret>
 S3_LOG_BUCKET: <my-bucket>
 S3_LOG_BUCKET_REGION: eu-central-1

apiVersion: v1
kind: Secret
metadata:
 name: db-secrets
type: Opaque
stringData:
 DATABASE_PASSWORD: <db-pass>

All resources were deploy on the same namespace, connection to db seems to be correct (it was populated correctly)

Relevant log output

2024-04-17 14:50:43 INFO i.m.l.PropertiesLoggingLevelsConfigurer(configureLogLevelForPrefix):113 - Setting log level 'DEBUG' for logger: 'io.airbyte.bootloader'
2024-04-17 14:50:45 INFO i.a.f.ConfigFileClient(<init>):105 - path /flags does not exist, will return default flag values
2024-04-17 14:50:48 ERROR i.m.r.Micronaut(handleStartupException):343 - Error starting Micronaut server: Error instantiating bean of type [io.airbyte.workers.storage.S3StorageClient]: region must not be blank or empty.
io.micronaut.context.exceptions.BeanInstantiationException: Error instantiating bean of type [io.airbyte.workers.storage.S3StorageClient]: region must not be blank or empty.
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2349) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2302) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1091) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.createBean(DefaultBeanContext.java:1071) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.BeanContext.createBean(BeanContext.java:157) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.airbyte.workers.storage.StorageClientFactory.get(StorageClient.kt:342) ~[io.airbyte-airbyte-commons-worker-0.57.3.jar:?]
	at io.airbyte.workers.config.CloudStorageBeanFactory.workloadStorageClient(CloudStorageBeanFactory.java:50) ~[io.airbyte-airbyte-workers-0.57.3.jar:?]
	at io.airbyte.workers.config.$CloudStorageBeanFactory$WorkloadStorageClient2$Definition.instantiate(Unknown Source) ~[io.airbyte-airbyte-workers-0.57.3.jar:?]
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2332) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2302) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2314) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3116) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:3018) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2979) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2753) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1752) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:89) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2165) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1328) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.airbyte.workers.workload.$JobOutputDocStore$Definition.instantiate(Unknown Source) ~[io.airbyte-airbyte-commons-worker-0.57.3.jar:?]
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2332) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2302) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2314) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3116) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:3018) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2979) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2753) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1752) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:89) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2165) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1328) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.airbyte.workers.temporal.check.connection.$CheckConnectionActivityImpl$Definition.instantiate(Unknown Source) ~[io.airbyte-airbyte-workers-0.57.3.jar:?]
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2332) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2302) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2314) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3116) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:3018) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2979) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2753) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1752) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:89) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2165) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1328) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.airbyte.workers.config.$ActivityBeanFactory$CheckConnectionActivities0$Definition.instantiate(Unknown Source) ~[io.airbyte-airbyte-workers-0.57.3.jar:?]
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2332) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2302) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2314) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3116) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:3018) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2979) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2753) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.findBean(DefaultBeanContext.java:1815) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.AbstractBeanResolutionContext.findBean(AbstractBeanResolutionContext.java:112) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.AbstractInitializableBeanDefinition.resolveOptionalBean(AbstractInitializableBeanDefinition.java:2309) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.AbstractInitializableBeanDefinition.findBeanForField(AbstractInitializableBeanDefinition.java:1908) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.airbyte.workers.$ApplicationInitializer$Definition.inject(Unknown Source) ~[io.airbyte-airbyte-workers-0.57.3.jar:?]
	at io.airbyte.workers.$ApplicationInitializer$Definition.instantiate(Unknown Source) ~[io.airbyte-airbyte-workers-0.57.3.jar:?]
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2332) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2302) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2314) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3116) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:3018) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2979) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2953) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.addCandidateToList(DefaultBeanContext.java:3568) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.resolveBeanRegistrations(DefaultBeanContext.java:3516) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.getBeanRegistrations(DefaultBeanContext.java:3486) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.getBeansOfType(DefaultBeanContext.java:1453) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.getBeansOfType(DefaultBeanContext.java:899) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.getBeansOfType(DefaultBeanContext.java:889) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.event.ApplicationEventPublisherFactory$2.lambda$$1(ApplicationEventPublisherFactory.java:215) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.core.util.SupplierUtil$1.get(SupplierUtil.java:47) ~[micronaut-core-4.3.14.jar:4.3.14]
	at io.micronaut.context.event.ApplicationEventPublisherFactory$2.publishEvent(ApplicationEventPublisherFactory.java:226) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.http.server.netty.discovery.NettyServiceDiscovery.onStart(NettyServiceDiscovery.java:54) ~[micronaut-http-server-netty-4.3.14.jar:4.3.14]
	at io.micronaut.http.server.netty.discovery.$NettyServiceDiscovery$Definition$Exec.dispatch(Unknown Source) ~[micronaut-http-server-netty-4.3.14.jar:4.3.14]
	at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:456) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext$BeanExecutionHandle.invoke(DefaultBeanContext.java:3903) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.aop.chain.AdapterIntroduction.intercept(AdapterIntroduction.java:84) ~[micronaut-aop-4.3.14.jar:4.3.14]
	at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:138) ~[micronaut-aop-4.3.14.jar:4.3.14]
	at io.micronaut.http.server.netty.discovery.NettyServiceDiscovery$ApplicationEventListener$onStart1$Intercepted.onApplicationEvent(Unknown Source) ~[micronaut-http-server-netty-4.3.14.jar:4.3.14]
	at io.micronaut.context.event.ApplicationEventPublisherFactory.notifyEventListeners(ApplicationEventPublisherFactory.java:266) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.event.ApplicationEventPublisherFactory$2.publishEvent(ApplicationEventPublisherFactory.java:226) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.http.server.netty.NettyHttpServer.fireStartupEvents(NettyHttpServer.java:638) ~[micronaut-http-server-netty-4.3.14.jar:4.3.14]
	at io.micronaut.http.server.netty.NettyHttpServer.start(NettyHttpServer.java:309) ~[micronaut-http-server-netty-4.3.14.jar:4.3.14]
	at io.micronaut.runtime.Micronaut.start(Micronaut.java:80) [micronaut-context-4.3.14.jar:4.3.14]
	at io.micronaut.runtime.Micronaut.run(Micronaut.java:328) [micronaut-context-4.3.14.jar:4.3.14]
	at io.micronaut.runtime.Micronaut.run(Micronaut.java:314) [micronaut-context-4.3.14.jar:4.3.14]
	at io.airbyte.workers.Application.main(Application.java:15) [io.airbyte-airbyte-workers-0.57.3.jar:?]
Caused by: java.lang.IllegalArgumentException: region must not be blank or empty.
	at software.amazon.awssdk.utils.Validate.paramNotBlank(Validate.java:180) ~[utils-2.23.17.jar:?]
	at software.amazon.awssdk.regions.Region.of(Region.java:155) ~[regions-2.23.17.jar:?]
	at software.amazon.awssdk.regions.Region.of(Region.java:151) ~[regions-2.23.17.jar:?]
	at io.airbyte.workers.storage.StorageClientKt.s3Client(StorageClient.kt:316) ~[io.airbyte-airbyte-commons-worker-0.57.3.jar:?]
	at io.airbyte.workers.storage.S3StorageClient.<init>(StorageClient.kt:223) ~[io.airbyte-airbyte-commons-worker-0.57.3.jar:?]
	at io.airbyte.workers.storage.$S3StorageClient$Definition.doInstantiate(Unknown Source) ~[io.airbyte-airbyte-commons-worker-0.57.3.jar:?]
	at io.micronaut.context.AbstractInitializableBeanDefinition.instantiate(AbstractInitializableBeanDefinition.java:774) ~[micronaut-inject-4.3.14.jar:4.3.14]
	at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2330) ~[micronaut-inject-4.3.14.jar:4.3.14]
	... 93 more

damian-aifinyo avatar Apr 17 '24 14:04 damian-aifinyo

Did you change the log4jConfig value in the values.yaml file?

marcosmarxm avatar Apr 18 '24 00:04 marcosmarxm

Thanks fore the hint! I found this on airbyte-commons/src/main/resources/log4j2-s3.xml

<!-- Note that logging to S3 will leverage the DefaultAWSCredentialsProviderChain for auth. -->
<Property name="s3-bucket">${sys:STORAGE_BUCKET_LOG:-${env:STORAGE_BUCKET_LOG:-}}</Property>
<Property name="s3-region">${sys:AWS_DEFAULT_REGION:-${env:AWS_DEFAULT_REGION:-}}</Property>

So, it is using the AWS_DEFAULT_REGION value, but is not set anywhere on the guide. I added this:

worker:
  extraEnv:
    - name: AWS_DEFAULT_REGION
      valueFrom:
        secretKeyRef:
          name: airbyte-logs-secrets
          key: S3_LOG_BUCKET_REGION

and it worked, but the next issue was the default value for the logs, it was trying to use this:

STORAGE_BUCKET_ACTIVITY_PAYLOAD: airbyte-storage
STORAGE_BUCKET_LOG: airbyte-storage
STORAGE_BUCKET_STATE: airbyte-storage
STORAGE_BUCKET_WORKLOAD_OUTPUT: airbyte-storage

Ignoring the values I set for the logs, I change that manually on the configMap airbyte-airbyte-env with my values and now is working.

again, thanks for the support!

damian-aifinyo avatar Apr 18 '24 07:04 damian-aifinyo

@damian-aifinyo Just faced the same issue and this config section helped to update these envs:

global:
####
  storage:
    type: "S3"
    bucket:
      activityPayload: <bucket_name>
      log: <bucket_name>
      state: <bucket_name>
      workloadOutput: <bucket_name>

silazare avatar Apr 26 '24 08:04 silazare

These steps help get past deploy, but in my scenario, any sync attempts fail with a 500 error whose response cites "java.lang.IllegalArgumentException: region must not be blank or empty."

I've used the Airbyte helm chart on and off for months, and every time it comes to install or update, it requires a willingness to dive deep into the source itself and debug the setup; the S3 installation instructions are horrible, the default overrides are buggy. It feels like one of those setups that's just supported enough to claim it exists, while also supported so poorly that users get pushed into the Cloud product.

apclapp avatar May 16 '24 16:05 apclapp

Both silazare and damian-aifinyo solutions got me moving again - thanks all

theblinkingusb avatar Jun 11 '24 21:06 theblinkingusb

Hi Team, any suggestion for adding the bucket path in the s3 configuration for external logging.

If bucket path added in below format

global:

storage: type: "S3" bucket: activityPayload: <bucket_name>/logs log: <bucket_name>/logs state: <bucket_name>/logs workloadOutput: <bucket_name>/logs

  provides an below error
  Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: The request signature we calculated does not match the signature you provided.

Shoban-ideas2it avatar Jun 27 '24 06:06 Shoban-ideas2it

In my case, I need provide for server as well, so like

global:
  storage:
    type: S3
    bucket:
      activityPayload: production-hm-airbyte
      log: production-hm-airbyte
      state: production-hm-airbyte
      workloadOutput: production-hm-airbyte
server:
  extraEnv:
    - name: AWS_DEFAULT_REGION
      valueFrom:
        secretKeyRef:
          name: hm-airbyte-secret
          key: AIRBYTE_LOG_S3_BUCKET_REGION
worker:
  extraEnv:
    - name: AWS_DEFAULT_REGION
      valueFrom:
        secretKeyRef:
          name: hm-airbyte-secret
          key: AIRBYTE_LOG_S3_BUCKET_REGION

hongbo-miao avatar Jun 28 '24 07:06 hongbo-miao

What if you are using an external Minio setup? The region is empty in the Minio configuration. What are the steps here?

djpirra avatar Sep 13 '24 09:09 djpirra