dataproc-templates icon indicating copy to clipboard operation
dataproc-templates copied to clipboard

Published 20 hours ago • verified publisher icon GoogleCloudPlatform

Hardening Pub/Sub to BigQuery template

Open franklinWhaite opened this issue 3 years ago • 2 comments

Testing templates with up to 5TB of data and up to 1 GBs throughput

franklinWhaite avatar Jan 31 '22 18:01 franklinWhaite

  1. Encountered BigQuery API limit error , increased the batch size to align with API Hit limit.
  2. Was able to test upto 750 Gb of data .
  3. Will publish the test document by the end of this week .

Raevat7 avatar Jan 18 '23 10:01 Raevat7

Hi @akhilkurapati12, Could you provide an update on this issue?

vanshaj-bhatia avatar Feb 23 '24 06:02 vanshaj-bhatia