incubator-xtable
incubator-xtable copied to clipboard
apache
Apache XTable (incubating) is a cross-table converter for lakehouse table formats that facilitates interoperability across data processing systems and query engines.
1. Test partitioned by date type (struct date type) and partition by it i.e no generated column. This would be useful to translate to formats like iceberg where there is...
Another related question: could we make type of `pendingCommits` consistent with type of `commitsToProcess`, i.e. use `COMMIT` instead of `Instant`? Would it break anything for Hudi? COMMITS uniquely identify a...
The spec id expected in `getTable` is expected to be spec as_of snapshot. Currently the one used is spec of an iceberg table, `iceTable.spec()` should return current spec id. This...
At this point, I do not know of any scenario which may require "in-flight" checks for Iceberg. The commit time (end of transaction) seems to be creating write serializability. A...
[LST-Bench](https://github.com/microsoft/lst-bench/) is a framework designed for running workloads focused on evaluating different aspects of table formats. It would be beneficial to incorporate LST-Bench into the CI pipeline of OneTable, which...
Once we're able to complete round trip testing (write in format A, convert to B, write in B, convert back to A), we need to assert that we are not...
For each pair of formats, we should have a test that can write in format A, sync to B, write an update in B, sync to A, and read the...