spark-acid

spark-acid copied to clipboard

Intermittent ORC File corruptions while using spark-acid write

3

Hi guys, we use spark to read from/ write to Hive Acid tables. We have been using spark-acid for both read and writes. And we started seeing the below error...

gowtamchandrahasa

Because of some history reasons, i should use hive version 1.2.1 or lower, and table set 'tbl_properties transactional = true' that cause Spark can not read data. In fact i...

leesanQAQ

INSERT OVERWRITE operation is updating the metastore information of ACID table to latest base directory even incase of failures

2

When trying to perform an INSERT OVERWRITE operation, the hive metastore information of a table are updated to point the latest base directory even in case of an error while...

srinikvv

PySpark Support

1

This is place holder issue to add support for PySpark.

citrusraj

I can not find the latest version on Marven, could anyone tell me where to find it?

sali2008

As the title suggests, the latest version cannot be found at https://mvnrepository.com/artifact/qubole/spark-acid?repo=spark-packages. Could somebody release it please ?

iamabug

NPE while reading Multiple Partitions

6

Hi Guys, we have integrated spark-acid library into our production pipeline and recently started facing an issue while reading data from a lot of partitions. Below is the stack trace...

adiu19

Issue 86 : Add support for Datasource V2 : ORC

2

maheshk114

We are using Merge on Hive Acid tables (non-bucketed) to maintain SCD Type-1 data with incremental updates from our sources. Over a period of time, regular merge statements on this...

srinikvv

spark-acid incorrectly reads/writes pre-Gregorian timestamps

3

In beeline: ``` 0: jdbc:hive2://localhost:10000> create table ts_acid (ts TIMESTAMP) stored as orc TBLPROPERTIES ('transactional' = 'true'); No rows affected (0.132 seconds) 0: jdbc:hive2://localhost:10000> insert into ts_acid values ('1200-01-01 00:00:00.0');...

bersprockets