Paola Pardo
Paola Pardo
[Deverloper's Guide]( https://github.com/Qbeast-io/qbeast-spark/blob/main/CONTRIBUTING.md) docs should point to the new version of Qbeast-Spark instead of 0.4.0.
Following the news on Delta and Spark releases we should update the` qbeast-spark` libraries to keep up to date with the latest features. This is the summary of the new...
Right now, if we have a table in Parquet or Delta and we want to transform it to Qbeast we can do two operations: - Full Rewrite of the table...
Right now, Revision Changes are being treated in three different parts: 1. `SparkRevisionFactory` -> creates a Revision with user configurations (`columnsToIndex`, `columnStats`...) 2. `OTreeAnalyzer` -> analyses the data and triggers...
Related to #166 . Qbeast-Spark should be compatible with latest versions of Delta Lake and Apache Spark, to benefit from any new features and major upgrades. The change to Delta...
The current implementation of the Qbeast Format on Spark does not support appending new data with a different indexed columns that the ones specified at the beginning. Since data is...
As described in this issue of Delta Lake project https://github.com/delta-io/delta/issues/1408, they are planning to get rid of version control and replace it for a set of features that must be...
Some multiple factors and parameters may influence writing and reading behavior, and right now there is no easy way to find which tuning is necessary. It takes a combination of...
This issue is to clarify the status of Delete Operation in Qbeast Spark library and which are the further steps on the roadmap. DELETE is a basic Data Management operation...
Table Formats encapsulate write actions into an Optimistic Transaction. Various processes could try to commit the info to the Transaction Log, but only one would succeed, making the others retry...