scio icon indicating copy to clipboard operation
scio copied to clipboard

saveAsBigtable doesn't work if ImpersonatedCredentials are set in BigtableOptions

Open rculbertson opened this issue 3 years ago • 0 comments

This is not a common use case, and we worked around it, but wanted to flag it as an issue.

When we set ImpersonatedCredentials in BigtableOptions, and pass this to saveAsBigtable, the job fails with this error:

Caused by: java.lang.NullPointerException at com.google.auth.oauth2.ImpersonatedCredentials.refreshAccessToken(ImpersonatedCredentials.java:239)

I think this is because ImpersonatedCredentials contains a transient field, so that field is null in the deserialized instance on the worker.

If we could pass a callback function to the BigtableOptions method withBigtableOptionsConfigurator on this line, we could create the ImpersonatedCredentials on the worker and avoid the serialization issue. Maybe saveAsBigtable could let users pass a callback function?

rculbertson avatar Apr 13 '21 22:04 rculbertson