Paola Pardo
Paola Pardo
**What went wrong?** During the development and cleaning of the software, we missed having a version control on each of the implementations. Without this track, in the main branch, we...
**What went wrong?** Right now, if the user **changes** the `cubeSize` for the same table, a new `Revision` is issued. https://github.com/Qbeast-io/qbeast-spark/blob/35c9f5698ee7ce01e1eaf467013805dfad63d7da/src/main/scala/io/qbeast/spark/table/IndexedTable.scala#L149 https://github.com/Qbeast-io/qbeast-spark/blob/35c9f5698ee7ce01e1eaf467013805dfad63d7da/src/main/scala/io/qbeast/spark/table/IndexedTable.scala#L134 https://github.com/Qbeast-io/qbeast-spark/blob/35c9f5698ee7ce01e1eaf467013805dfad63d7da/src/main/scala/io/qbeast/spark/table/IndexedTable.scala#L138 But having different _desired cube sizes_ across...
The problem with the current implementation is that the files from the `ReplicatedSet` are still visible for the delta log. So, if someone wants to read the table as `delta`...
Schema evolution is a feature of Delta Lake that allows users to easily change a table’s current schema to accommodate data that is changing over time. Most commonly, it’s used...
Now we only support those two operations on scala spark-shell. We need to buld a library for python as well to instantiate QbeastTable from a notebook or pyspark. Delta solution:...
## Description Adds feature #102 The idea is to have a way of converting an already existing table (either written in `Parquet` or `Delta`) into the `Qbeast` Format. To do...
Right now, when trying to create a Qbeast Table without schema, the following exception is raised: ```scala spark.sql("CREATE TABLE t USING qbeast LOCATION '/tmp/test'") org.apache.spark.sql.AnalysisException: Trying to create an External...
Within the past weeks, Delta Lake released its version 3.1.0 with some exciting opportunities for integrations with the ecosystem. Some of the highlights are: - Preview of [Liquid Clustering](https://github.com/delta-io/delta/issues/1874). Now...
Right now, the filtering works as follows: 1. Apply Delta Filters for Staging Area files. 2. Apply Qbeast Filters on all the rest. 3. Union both sets of files. We...
Qbeast Format allows indexing multiple columns to improve the file layout. Right now, the user should explicitly assign the set of columns that she/he wants to index through the `columnsToIndex`...