Not able to make this work with minio!

[ash-lyrid]

Hi Team,

I am trying to make this utility work with minio and having some issues. Any help or pointers to resolve this is highly appreciated.

I have installed the minio via docker on the same server where my airflow is installed. I am able to access minio properly via the publicip:9001. I then created a bucket by the name dags and able to upload files.

Now I want to sync the dags volume of the airflow container with the minio. For test purposes I am trying to run it with the following arguments but always getting error. I am having hardtime uderstanding how to fix this.

objinsync pull --once --disable-ssl --s3-endpoint http://localhost:9000 s3://dags/*.* ~/Projects/airflow/dags

{"level":"info","ts":1657540886.0054412,"caller":"objinsync/main.go:87","msg":"SENTRY_DSN not found, sentry integration disabled."}
{"level":"info","ts":1657540886.009082,"caller":"objinsync/main.go:142","msg":"Pulling from s3://dags/*.* to /home/lyridadmin/Projects/airflow/dags..."}
{"level":"info","ts":1657540886.0091188,"caller":"objinsync/main.go:126","msg":"Pull started."}
ERROR: failed to pull objects from remote store: Failed to detect AWS region: EC2MetadataRequestError: failed to get EC2 instance identity document
caused by: RequestError: send request failed
caused by: Get "http://169.254.169.254/latest/dynamic/instance-identity/document": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Tried with this as well - but getting the same error as above. objinsync pull --once --disable-ssl --s3-endpoint http://publicip:9000 s3://dags/*.* ~/Projects/airflow/dags

Edit: Also, how do I put the details of the accessKey and SecretKey of the Minio?

[heavenlxj]

i have the same question for this, so anyone can give some helps?

[heavenlxj]

hi @ash-lyrid i have fixed the code including the credential AK/SK for the minio server, and will send a PR soon

[heavenlxj]

@ash-lyrid hi, i fixed the code missing the credential with AK/SK for minio server, and i will send a PR

[heavenlxj]

attached the PR: https://github.com/scribd/objinsync/pull/22

@ash-lyrid by the way, you need to set the env "export AWS_REGION="us-east-1" it worked for me.