scio
scio copied to clipboard
saveAsBigtable doesn't work if ImpersonatedCredentials are set in BigtableOptions
This is not a common use case, and we worked around it, but wanted to flag it as an issue.
When we set ImpersonatedCredentials
in BigtableOptions
, and pass this to saveAsBigtable
, the job fails with this error:
Caused by: java.lang.NullPointerException at com.google.auth.oauth2.ImpersonatedCredentials.refreshAccessToken(ImpersonatedCredentials.java:239)
I think this is because ImpersonatedCredentials
contains a transient field, so that field is null in the deserialized instance on the worker.
If we could pass a callback function to the BigtableOptions
method withBigtableOptionsConfigurator
on this line, we could create the ImpersonatedCredentials
on the worker and avoid the serialization issue. Maybe saveAsBigtable
could let users pass a callback function?