DataflowJavaSDK icon indicating copy to clipboard operation
DataflowJavaSDK copied to clipboard

Published 20 hours ago • verified publisher icon GoogleCloudPlatform

Using Service Account with dataflow

Open seehans opened this issue 7 years ago • 2 comments

I am trying to use a dataflow job with a service account. I am using the GcpOptions flags --serviceAccountKeyfile="dataflow-service-account.p12" --serviceAccountName="dataflow"

I am getting following error: Unable to verify that GCS bucket exists.com.google.cloud.dataflow.sdk.util.DataflowPathValidator.verifyPathIsAccessible(DataflowPathValidator.java:84) com.google.cloud.dataflow.sdk.util.DataflowPathValidator.validateOutputFilePrefixSupported(DataflowPathValidator.java:63) com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.fromOptions(DataflowPipelineRunner.java:274)

I would like to use service acoount credentials against application default credentials.

seehans avatar Feb 28 '17 22:02 seehans

It works if i give following value to serviceAccountName flag: --serviceAccountName="[email protected]"

Documentation is misleading, we are actually giving "Service account ID" value to serviceAccountName flag and not "Service account name" .

seehans avatar Mar 01 '17 16:03 seehans

I'll reopen to clarify the documentation.

Thanks @seehans! Also, if you are interested, feel free to contribute a documentation fix to the Apache Beam codebase: https://github.com/apache/beam.

davorbonaci avatar Mar 01 '17 22:03 davorbonaci