Create BigQuery Sink

[anoopj]

Build a BigQuery sync will allow customers to incrementally sync from table formats to BigLake Managed Tables (and BQ native tables).

The implementation of the sink will be different than other sinks:

1. Instead of metadata translation, it will invoke a LOAD statement in BigQuery.
2. It can only support added files and not removed files.

@the-other-tim-brown: Do you think this will be useful?

[the-other-tim-brown]

It's an interesting use case and could work with our existing framework. I'm curious if there are other tools that already do this though.

[anoopj]

I am not aware of any existing tooling. BigQuery LOAD jobs support only file formats (Parquet/ORC/Avro et al) - open table formats are not supported.