ApiError: Authentication unknown error

[rafelis1997]

Environment details

• OS: GKE
• Node.js version: 18
• npm version: 10.7.0
• @google-cloud/bigquery version: 7.7.0

We are trying to insert data on BigQuery through this library but we are facing this error. Our api is running on GKE and we are using Workload Federation Identity as the authentication factor. We are able to create the dataset_id correctly on the pod startup, this suggests the problem could not be related to authentication besides the error tell us this.

From my search i've found out that firebase node library has this happening too

[alvarowolfx]

hey @rafelis1997 thanks for the report. Can you share more details on how are you setting up authentication when instantiating the client ? I might be missing something here, but afaik, Workload Federation Identity is used when accessing GCP services, outside of GCP (like using AWS or Azure). Ref https://github.com/googleapis/google-auth-library-nodejs/tree/main?tab=readme-ov-file#workload-identity-federation

When I personally used GKE in the past, I was used to use the service account from the VM on the GKE cluster, which should work automatically if the GKE cluster default service account has access to BigQuery. This is describe here: https://cloud.google.com/bigquery/docs/authentication#on-gcp

But you can also create a custom service account for your application, add it to the GKE cluster as a K8S Secret and mount it on your application. You can load as a file or use environment variables. This section on the google auth lib might help: https://github.com/googleapis/google-auth-library-nodejs/tree/main?tab=readme-ov-file#loading-credentials-from-environment-variables

In the meantime I'm reading more about Workload Identity Federation and seeing what kind of issues are happening on other node libraries as you reported and see what can be done. If you can provide more information on your use case, that might help.

[rafelis1997]

Sadly, the default service account not worked for us, our pods were receiving Access Denied error when creating the dataset_id. After grant permissions to the default GKE service account we solved this issue. But now we are facing this error for some reason

[alvarowolfx]

@rafelis1997 Can you try the option second option that I gave of creating a separate service account ? Then you can upload it to the cluster as a secret, mount to the pod and use that service account for auth. Good thing is that is bit easier to test locally too, as you can create a service account on the dev env and test with it on your machine and later you can create the production service account and upload to the GKE cluster.

Also, you haven't provided more information on the Workload Identity Federation part, I just want to make sure that I'm providing the right recommendation for your set up.

[rafelis1997]

accordingly to Google documentation Workload Identity federation is also used as authenticator to GKE using cloud API's on pods. https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity

I can try this extra step yes

[alvarowolfx]

@rafelis1997 have you managed to get things working ? I'll close the issue if no extra work is needed here.

[rafelis1997]

Yes you can close it

Em qua., 10 de jul. de 2024 14:30, Alvaro Viebrantz < @.***> escreveu:

@rafelis1997 https://github.com/rafelis1997 have you managed to get things working ? I'll close the issue if no extra work is needed here.

— Reply to this email directly, view it on GitHub https://github.com/googleapis/nodejs-bigquery/issues/1363#issuecomment-2221080405, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOOZJJ7VN7ZUB5BVYQ7SDADZLVVRXAVCNFSM6AAAAABHWLWFKSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRRGA4DANBQGU . You are receiving this because you were mentioned.Message ID: @.***>