dataproc-templates

dataproc-templates copied to clipboard

Eg: gcs.spanner.input.location=gs://dataproc-templates/data/avro/empavro gcs.gcs.input.location=gs://dataproc-templates/integration-testing/gcstogcs/csvtoavro/csv/cities.csv gcs.bigtable.input.location=gs://dataproc-templates/data/csv/GCSToBigTable_cities.csv gcs.bigquery.input.location=gs://dataproc-templates/integration-testing/gcstobigquery/cities.csv text.bigquery.input.location=gs://dataproc-templates/data/deflate should be checked in java/.ci/ Then in `stage('Reset Resources')` of JenkinsFile it should be overwritten before running the tests.

shashank-google

[Publishing] [Java] Blog for WordCount example

1

Publish a blog for running the [WordCount example](https://github.com/GoogleCloudPlatform/dataproc-templates/blob/main/java/src/main/java/com/google/cloud/dataproc/templates/word/WordCount.java) using Dataproc Serverless (Java)

violetautumn

Currently we are using export JARS option to provide spark bigquery connector jar file. It is available through maven central [repository](https://mvnrepository.com/artifact/com.google.cloud.spark/spark-bigquery-connector-common). Update pom.xml file to be available from there.

rajc242

Our spark bigquery connector is supporting append and overwrite mode based on [documentation](https://github.com/GoogleCloudDataproc/spark-bigquery-connector?tab=readme-ov-file#writing-data-to-bigquery). We need to thoroughly test and validate various scenarios and update README.md file for writing to BigQuery...

rajc242