qbeast-spark
qbeast-spark copied to clipboard
Unify Create Qbeast Table in one step
Right now, the code to create a Qbeast Table without data (no SAVE AS), is done in two steps:
- Create an empty data frame and save it in the Location.
- Convert the Delta Table to the Qbeast Table with
ConvertToQbeastCommand.
It is executed like that because we do not have access to methods in the CreateDeltaTableCommand, nor can we delegate the creation to that object. Otherwise (if we call directly CreateDeltaTableCommand), the table would be created in the Catalog as "delta", and the whole operation would fail.
The idea is to do both in the same transaction.
Perhaps it is a good moment to build a CreateQbeastTableCommand to centralize the process.
We need a design document for this issue to understand clearly the implications of the change.