Fernando Velasquez

Added Spark Source interface

1

This interface will be used to create Spark RDDs and RDDCollection entities from Spark programs. This will be useful when integrating the Spark BigQuery Connector for BQ Pushdown execution https://github.com/GoogleCloudDataproc/spark-bigquery-connector

Added support for Snappy compression in Avro records for BigQuery Pushdown jobs

2

Note that a table is uploaded using the Json format (when the schema contains Datetime fields), no compression will be used. Most of the logic implemented in this classes is...

Engines can use a new capability to signal that they support multiple input relational transform stages. The Spark SQL Engine is able to quickly combine input stages (using the dataframe...

This avoids the possibility of the log reader being stuck in a loop where log entries cannot be processed.

Cherry Pick of https://github.com/data-integrations/wrangler/pull/627 into 4.9 (CDAP 6.9)