delta
delta copied to clipboard
delta-io
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
## Description Adds a test parallelization option to run Delta tests on different JVM addressing #1128: - Implements a generic abstraction called `GroupingStrategy` and a default implementation `SimpleHashStrategy` that uses...
## Description (This PR is on top of delta-io/delta#1257) Support following SQL queries on Delta tables to allow reading from a specific version ``` SELECT * FROM default.people10m VERSION AS...
## Description Light refactor of python codebase with best practice, strings are escaped and some minor code tweaks. ## How was this patch tested? Locally tested using existing tests and...
This is more of a discussion as opposed to an issue. If we interleave bits, 0 will be always be at the top (so to speak). In your code, you...
## Feature request ### Overview We saw a few reported issues due to missing configs. We can add a check in DeltaLog to verify the following Spark confs. ``` --conf...
## Bug ### Describe the problem We were happy to see that OSS delta lake supported compaction, so we upgraded and tried to optimize our current tables. However, we observed...
## Feature request ### Overview Delta Lake does not resolve variables for `VACUUM` and `OPTIMIZE` commands, e.g. the following script fails ``` SET tbl=table_abc; OPTIMIZE ${tbl}; ``` ### Motivation Variable...
## Feature request ### Overview ### Motivation ### Further details ### Willingness to contribute The Delta Lake Community encourages new feature contributions. Would you or another member of your organization...
## Feature request ### Overview This [commit](https://github.com/delta-io/delta/commit/d90f90b6656648e170835f92152b69f77346dfcf) added 3 new TahoeFileIndex's for the Change Data Feed feature currently being developed for Delta Lake. These are: `CdcAddFileIndex`, `TahoeChangeFileIndex`, and `TahoeRemoveFileIndex`. These...
## Description Support OptimizeWrite described in https://docs.databricks.com/delta/optimizations/auto-optimize.html#how-optimized-writes-work Fixes #1158 If OptimizeWrite is enabled, inject `OptimizeWriteExchangeExec` on top of the write plan and remove `ShuffleExchangeExec` or `CoalesceExchange` operation at the top...
