AWS Cross Account access

[kostyaplis]

We are looking for a cross account access to Athena. I.e. our Metabase instance resides in account A, while Athena, S3 buckets and Glue Catalogs in account B within the same organisation.

While it seems possible to grant cross-account access to Athena itself, I think cross account assume_role supported by metabase-athena-driver would be desired solution.

Latest SimbaAthenaJDBC (2.0.21) doesn't provide any implementation for STSAssumeRoleSessionCredentialsProvider or STSAssumeRoleWithWebIdentitySessionCredentialsProvider. So potential solution would be to have custom credentials providers for those?

I've tested custom credentials providers approach following cross-account access scenario in AWS blog post. Confirm that it works as expected in SQLWorkbench with latest SimbaAthenaJDBC. However, it fails in Dbeaver with following errors: class "com.amazonaws.auth.AWSCredentialsProviderChain"'s signer information does not match signer information of other classes in the same package OR Could not initialize class com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClientBuilder

Next step I would test CustomIAMRoleAssumptionCredentialsProvider with metabase-athena-driver, but getting some issues building current master: Caused by: java.lang.RuntimeException: No such var: m/update-existing Full log attached: lein.log

Please do let me know if any of above does make sense. At the same time I would try to get the project compiled, so I'd be able to test any changes. Any advise is highly appreciated.

[dacort]

Hi @kostyaplis - A couple questions.

1. Can you just use IAM credentials when setting up Metabase? Metabase wouldn't necessarily need cross-account access...it just needs the right credentials to connect to the Athena endpoint.
2. Athena just announced officially-supported cross-account access the other day. https://docs.aws.amazon.com/athena/latest/ug/data-sources-glue-cross-account.html You might be able to try setting that up.

re: the compilation error, can you confirm that you installed Metabase core? https://github.com/metabase/metabase/wiki/Writing-a-Driver:-Packaging-a-Driver-&-Metabase-Plugin-Basics#installing-metabase-core-locally

[kostyaplis]

Hi @dacort, thanks for the prompt reply! I'm able to configure Athena driver with IAM credentials and Athena cross-account access following links as per your advise. In case of someone come across, there is a list of configurations I had to apply to make it working: Borrower account:

1. Create Athena workgroup with engine v2
2. Create query results S3 bucket
3. Create AWS IAM policy
   • Allow glue actions in Owner account (link - Step 1a)
   • Allow athena and glue actions in Borrower account
   • Allow s3 actions on query results bucket in Borrower account
   • Allow s3 actions on each queryable bucket in Owner account
4. Register cross-account Glue Data catalog link

Owner account:

1. Create Glue Data catalog permissions link - Step 1b
2. Create S3 Bucket Policy for EACH bucket queryable from Borrower account link
   • If bucket objects owned by other AWS account (i.e. AWS service logs, etc), object ownership have to be adjusted.

Above is working for me and I'm able to query the data. However, configuration and maintenance seems quite complicated comparing to assumable-role concept, where I would have a single policy in Borrower account that allows to assume the role in Owner account. And a single role with attached policy granting all required permissions in Owner account. So I'm going to check assumable-role concept with SQL Workbench first. Then will re-iterate on custom credentials provider and let you know....

---

As for compilation issues. Yes, I was following Contributing guide. lein install-for-building-drivers on metabase repo had finished successfully with Installed jar and pom into local repo. message. However, still stuck with Caused by: java.lang.RuntimeException: No such var: m/update-existing error message during build.

[dacort]

@kostyaplis Are you compiling against Metabase 0.38 or 0.39? I see that 0.39 refactored the type system and I haven't tested this driver against 0.39 yet, so that could be the issue. I'll try to take a look next week - this week is pretty packed. :)

[kostyaplis]

Hi @dacort, Sorry for dummy questions, it was my first Clojure project to build. So issue is in my local environment. Still didn't figured out the root cause. But I can build just fine in docker container or clean EC2 instance. Also confirm that it compiles against metabase v0.39.2

I've added custom credentials provider that allows me to assume a role. Confirm it works with cross-account assumable roles. As of now I'm actively testing it and will share my experience. Also raised a question to AWS Support regarding role assumption support by SimbaAthenaJDBC.

Code I'm working on lives here. Will create a PR once the solution is proven.

[gpodevijn]

Hi @dacort and @kostyaplis,

I tested @kostyaplis's branch and I can confirm that it works perfectly.

Is there any chance that this code gets merged into master?

Regards, Gaetan

[kostyaplis]

@gpodevijn unfortunately it haven't been finished AFAIR it doesn't support temp credentials refresh (

[dacort]

Closing as part of cleanup now that Athena is officially supported by Metabase. Any future issues can be asked about on their forum or with a detailed bug report.