scio
scio copied to clipboard
Support clustering in saveAs..BigQueryTable
I am trying to write to a clustered table using saveAsTypedBigQueryTable
and getting:
<table> is configured to use TableDestinationCoderV2. Set withClustering() on BigQueryIO.write() and, if you provided a custom DynamicDestinations instance, override getDestinationCoder() to return TableDestinationCoderV3.
I only hit this when I am processing many partitions at once. I see that the SCIO API doesn't expose the withClustering option.
https://beam.apache.org/releases/javadoc/2.31.0/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.Write.html#withClustering-com.google.api.services.bigquery.model.Clustering-