codertimu
codertimu
@reuvenlax What's the purpose of using `withAutoSchemaUpdate(true)` with `STORAGE_API` methods? According to the documentation: _[withAutoSchemaUpdate(boolean autoSchemaUpdate)](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.Write.html#withAutoSchemaUpdate-boolean-) If set to true, it enables automatic detection of schema updates in BigQuery tables....
Any updates on this? Python api has this but Java api lacks it. Can't it be prioritised?
Interesting, I used 2.43 with the appropriate constructor that takes messageId and orderingKey but no messageId and orderingKey was in Pubsub. I am using dataflow runner. Could it be related...
I used `PubsubMessageWithAttributesAndMessageIdAndOrderingKeyCoder` on the PCollection of Pubsub messages but still `messageId`(I don't care it) and `orderingKey` is null. I used it as follows: ```java results.setCoder(new PubsubMessageWithAttributesAndMessageIdAndOrderingKeyCoder()) .apply("Write Result updates...
@egalpin registerCoder does not exist. Do you mean like this: ```java cr.registerCoderForClass(PubsubMessage.class, new PubsubMessageWithAttributesAndMessageIdAndOrderingKeyCoder()); ```
Tried above and this: ```java cr.registerCoderForType(TypeDescriptor.of(PubsubMessage.class), new PubsubMessageWithAttributesAndMessageIdAndOrderingKeyCoder()); ``` but nothing changed.
It is unbounded but I was using the test pipeline for the last few examples I tried here. I didn't attempt to run in dataflow.
@egalpin just to clarify, in my test pipeline, PubsubIO is not involved at all. Even in a simple ptransform, the orderingkey and messageid are not preserved in the pcollection.
I confirm that the issue persists with the dataflow runner as well.