spring-batch-extensions icon indicating copy to clipboard operation
spring-batch-extensions copied to clipboard
verified publisher icon spring-projects

Investigate possibility to support Parquet

Open dgray16 opened this issue 4 years ago • 2 comments

https://hadoop.apache.org/ https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-parquet https://cloud.google.com/bigquery/docs/exporting-data#parquet_export_details

  • Avro (implemented)
  • Protobuf
  • ...

dgray16 avatar May 03 '21 05:05 dgray16

Spring had a Spring for Apache Hadoop module that was built upon Spring Batch. I don't think we should revive that kind of functionality here. If there is interest in that project still, you should take it up with that project: https://spring.io/projects/spring-hadoop

mminella avatar May 03 '21 13:05 mminella

@mminella thanks for the update, but this issue is created to investigate possibility to add support of Parquet data type for BigQuery Spring Batch module. Will change topic name to make it more clear.

dgray16 avatar May 04 '21 05:05 dgray16