scio icon indicating copy to clipboard operation
scio copied to clipboard

Query being executed against wrong project

Open siddjain opened this issue 2 years ago • 0 comments

I am following the getting started guide of scio. When I run:

runMain com.spotify.scio.examples.WordCount --project=my-gcp-project --runner=DataflowRunner --zone=us-west1 --input=gs://apache-beam-samples/shakespeare/kinglear.txt --output=gs://XXX/scio/wordcount

Two things happen:

  • first of all, it does not run com.spotify.scio.examples.WordCount. Instead it runs com.spotify.scio.examples.extra.TypedStorageBigQueryTornadoes.scala. I can't figure out why.
  • and then this code BigQueryType.fromQuery("SELECT tornado, month FROM [bigquery-public-data:samples.gsod]") results in a call to BQ but executes against the wrong project.

How can I fix this?

I have run gcloud auth application-default login. I have also run gcloud config set project <project-id>. I have verified through gcloud init. I have a Java program and that correctly targets the correct project but not scio. I also looked at ~/.config/gcloud/application_default_credentials.json but scio is executing against wrong project.

siddjain avatar Jun 17 '22 21:06 siddjain