connect icon indicating copy to clipboard operation
connect copied to clipboard

New output component for BigQuery for the new Write API

Open Locustv2 opened this issue 2 years ago • 1 comments

As the current BigQuery component is still experimental, it would be good to replace it with a new component using the Write API from Big Query: BigQuery Write API explained: An overview of the Write API

The advantage of using this api is that it uses Protobuf data. Which means it's faster as it will be moving binary data around.

Here are some additional material: Learn how to stream JSON data into BigQuery using the new BigQuery Storage Write API

Locustv2 avatar Mar 10 '22 14:03 Locustv2

The link appears to be dead! But I would be interested in this too. New link: https://cloud.google.com/bigquery/docs/write-api

irfanhabib avatar Jun 20 '22 10:06 irfanhabib