DataflowJavaSDK

DataflowJavaSDK copied to clipboard

still digging into exactly why this happens, but meanwhile this should be a checked exception, probably some kind of IOException. This is an external condition, not a bug in the...

elharo

Reading multiple files and need the names in processing

3

Hi, I have a requirement where I am reading 2K files and want to parse and validate the same. In the process of validating I need to know the file...

parwaniravi248

and re-add the Java8 override for examples-java8 module

dhalperi

_TL;DR: Existing Maven projects that use the recommended version range for the Google Cloud Dataflow SDK for Java may soon pick up a new version that causes breakages at the...

francesperry

filling default.string() with getoutput()

1

I am not a Java guru I am having a problem to fill the values on this script on the description level , on how to fill the value on...

demousteven

Store OOM dump on GCS

1

Is there a way to have pipelines with `dumpHeapOnOOM` enabled store the dumps on GCS instead of (or in addition to) the disk? There is `setSaveProfilesToGcs` for performance profiles, so...

pelletier

TextIO - add "allow empty" flag

1

Since SDK 1.9: Changed FileBasedSource to throw an exception when reading from a file pattern that has no matches. Pipelines will now fail at runtime rather than silently reading no...

rantibi

Using Service Account with dataflow

2

I am trying to use a dataflow job with a service account. I am using the GcpOptions flags --serviceAccountKeyfile="dataflow-service-account.p12" --serviceAccountName="dataflow" I am getting following error: Unable to verify that GCS...

seehans

Files from classpath are not properly resolved when classpath JAR contains META-INF with references to other dependencies.

3

Hello, Intelij has a feature called dynamic.classpath which is used when the amount of items in the classpath required to launch the JVM+program exceed the maximum line size allowed by...

metabrain

Dataflow SDK and Google Cloud Java SDK conflict

4

Google Cloud SDK which includes all the libraries from Cloud Storage to Cloud PubSub have conflict showing `Exception in thread "main" java.lang.NoClassDefFoundError: com/google/protobuf/GeneratedMessageV3 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:763) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)...

vaibhavsw