airflow icon indicating copy to clipboard operation
airflow copied to clipboard

Add additional authentication for Google Cloud Operators

Open ChloeSheasby opened this issue 2 years ago • 3 comments

Description

Currently, the only way to authenticate with Google Cloud operators is through defining a file path to the Google Cloud Credentials. This is not always a safe option, and credentials can be acquired in different ways through the Workload Federated Identity service in Google Cloud Platform.

This can be done in the following:

  • Request a token from your external service (defined in your Federated Identity)
  • Provide the following items as the payload to this api endpoint: https://sts.googleapis.com/v1/token
    • subject_token: the token from the previous request
    • grant_type: "urn:ietf:params:oauth:grant-type:token-exchange"
    • requested_token_type: "urn:ietf:params:oauth:token-type:access_token"
    • scope: "https://www.googleapis.com/auth/cloud-platform"
    • subject_token_type: "urn:ietf:params:oauth:token-type:jwt"
  • Provide the following items to this api endpoint: https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/:generateAccessToken
    • Headers: use the token from the previous request in the "Authorization" section
    • Payload:
      • scope: ["https://www.googleapis.com/auth/cloud-platform"]
  • Use the token from the last request as the authentication.

Use case/motivation

I would like to be able to authenticate the Google Cloud operators through the Workload Federated Identity instead of providing the file path to credentials. This would help me to be able to use Airflow's native Google Cloud operators instead of making custom operators.

Related issues

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

ChloeSheasby avatar Nov 27 '23 19:11 ChloeSheasby

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

boring-cyborg[bot] avatar Nov 27 '23 20:11 boring-cyborg[bot]

Hello! I would be eager to implement this feature.

dybolo avatar Apr 25 '24 12:04 dybolo

@VladaZakharova and her team might be the right people to get feedback from on it

potiuk avatar Apr 25 '24 12:04 potiuk