DataflowJavaSDK

DataflowJavaSDK copied to clipboard

Angels

1

Angel

lisa1976

Release 2.4.0

1

Angels

lisa1976

Since porting to 2.1.0, Dataflow is leaving Datasets/Tables behind in BigQuery

9

Since porting to 2.1.0, Dataflow is leaving Datasets/Tables behind in BigQuery when the pipeline is cancelled or when it fails. We've been on 1.8.0/1.9.0 previous to this, and we've never...

polleyg

Error 409 from BigQuery when using in template

16

I'm doing a simple batch-job, that I'm implementing as a template. It is supposed to read data from BigQuery. Everything works fine on the first run, but all subsequent executions...

sduck

BigQueryIO need underlying "bigquery.tables.get permission" access to query a view

11

I am running a BigQueryIO like BigQueryIO.Read.usingStandardSql().fromQuery("SELECT * FROM `foo-bar-123456.category_view.markets` LIMIT 1000") but seeing this error, "The user [email protected] does not have bigquery.tables.get permission for table foo-bar-123456.category.markets" which `category` is...

wli600

Help

lisa1976

Enable for setting configuration.load.maxBadRecords within Dataflow

1

evertonjlima

Enable worker-level profiling of Dataflow Jobs

9

For both working on the SDK and building Dataflow pipelines, it would be useful if there was an easy way to get profiles from the execution of code on the...

bjchambers

PCollection<Void> instead of PDone

2

I'm not expecting this to be done. But I do want to highlight the UseCase I have for this. My environment is as follows. 1. I only allow templates to...

brucedeen

Cannot launch template with repeated parameters

4

It is not possible to launch a templated dataflow that accepts an option in the form of `ValueProvider`, despite support for Lists within the `ValueProvider` class. I'm not sure if...

dadrian