scio
scio copied to clipboard
Support BigQuery Load jobs for batch loading
From beam documentation
When you specify load jobs as the insertion method using BigQueryIO.write().withMethod(FILE_LOADS).
Scio should also give users the possibility to use files to load data to BigQuery
@RustedBones -- not sure I understand your issue here.
Is there something you are trying to do that you are not able to ( asking about functionality, not specific syntax/methods ).
Ah, based on your other issue ( https://github.com/spotify/scio/issues/4333 ), it looks like you're thinking Scio would benefit from BeamIO.Write Methods --> https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.Write.Method.html ?
That's correct. At the moment those are not exposed in the Scio API. Both tickets have been opened after user's request, feel free to assign yourself if you want to work on them.
I will need to dig into specifics a bit more -- as don't want to assign myself, until sure that have the ability to see it through relatively quickly ( to not be a holdup/blocker )