Guy Khazma
Guy Khazma
This PR fixes a minor issue with the condition checking for GET requests without range reads. In addition, since the reader filters only GET requests, I am not sure that...
Need to verify if when using datasource v2 the metadata is being read twice. This is currently handled in the tests by multiplying by 2 the number of files expected...
Currently the catalog table tests are running only for Parquet formats. We should add tests that will run for csv, json, avro etc.
This issue will track the progress of decoupling xskipper from Spark. Xskipper currently works solely with Spark and can not be used out of the box with other engines. This...
[SPARK-7768](https://issues.apache.org/jira/browse/SPARK-7768?jql=project%20%3D%20SPARK%20AND%20text%20~%20%22udt%20public%22) finally made the UDT API public. We can use this instead of the workaround we had so far using `ParquetMetadataStoreUDTRegistrator` to expose the `UDTRegistration` class.
The current API doesn't enable to add a new index without first dropping the existing indexes and the collecting all of the indexes again. This API will be useful for...
[Hudi's Copy-on-Write](https://hudi.apache.org/docs/querying_data.html#spark-datasource) optimized mode is using the standard Spark's parquet read code path. This means we should be able to support it as we already support skipping over parquet datasets...
[Delta Lake](https://delta.io) implementation is using a custom `FileIndex` implementation called [`TahoeFileIndex`](https://github.com/delta-io/delta/blob/master/src/main/scala/org/apache/spark/sql/delta/files/TahoeFileIndex.scala). Therefore, we can support skipping for Delta Lake in a similar way that we support skipping for the built...
An initial proposal that will enable to support Iceberg is detailed [here](https://docs.google.com/document/d/11o3T7XQVITY_5F9Vbri9lF9oJjDZKjHIso7K8tEaFfY/edit#).
The current release CI is working but can't push the version commit change to the repo. For now this can be done manually (the rest of the release process is...