DataflowTemplates
DataflowTemplates copied to clipboard
Latest with beam 2.29 not working for PubSubToBigQuery
Would be great with a new release with beam-2.31.
https://issues.apache.org/jira/browse/BEAM-12356
Basically, my jobs are getting thousands and thousands of exceptions and finally there is an OutOfMemoryError.
Downgrading to an older with beam-2.27 works. I have also built and tried with 2.31 and I have verified that that also works as expected.
I will work on it. I have been receiving a lot of errors too.
Fix was fixed on the Beam side, and we are aiming to be at the latest version (right now, 2.44.0).
--
This issue has been stale for some time now. Please reopen it if there is a follow up or any related questions.