Dan Halperin

I think this should probably be added to the Datastore source/sink builders rather than to a global pipeline options. This would encapsulate configuration in the right place. Also, could you...

My understanding is that this is a fundamental limitation of GCS's encoded-type format. 1. TextIO.Read uses file extension to determine whether a file is compressed, and `.txt` says it is...

I think you're right @rfevang . I'll try to follow up. Right now, Google Cloud Storage just looks like a filesystem that lies to us about its file size, but...

Leaving this bug open to track.

Hi Dan, Thanks for this report. It indeed looks like we did not implement handling for a jar with a `Class-Path` manifest, which is now used in IntelliJ 15+. We'll...

For now, you can either turn off dynamic classpath or use the `--filesToStage` option to manually pass in an explicit list of the jars to stage. You can also use...

We're investigating this internally -- the key issue right now is a mismatch between what the BigQuery UI does and what Dataflow does. The BigQuery UI can get information via...

Hi Graham, The vast majority of customers will not be affected, because the default `TextIO.Read.from("filepattern")` will automatically notice `.gz` files and decompress them. Affected jobs are only those using version...

Cloud Dataflow SDK for Java 1.6.1 has been released with a fix to this issue. - GitHub release: https://github.com/GoogleCloudPlatform/DataflowJavaSDK/releases/tag/v1.6.1 - [Release Notes on `cloud.google.com`](https://cloud.google.com/dataflow/release-notes/java#161) - On [Maven Central](https://search.maven.org/#artifactdetails%7Ccom.google.cloud.dataflow%7Cgoogle-cloud-dataflow-java-sdk-all%7C1.6.1%7Cjar) See [`Downloads`](https://cloud.google.com/dataflow/downloads)...

Hi, I wonder if you have pre-created the table in BigQuery? If so, you might delete the table from the BigQuery web UI and then try again? Thanks, Dan