[Question][Standalone] How to add `constraints` when creating tables with delta-standalone?

[JaySardhara]

Question

Which Delta project/connector is this regarding?

• [ ] Spark
• [x] Standalone
• [ ] Flink
• [ ] Kernel
• [ ] Other (fill in here)

Overview

How can we handle adding constraints when tables are created using delta-standalone? For example: In delta-spark we add constraints shown below. spark.sql("Alter table delta.my_table7 add constraint age_less_than check (age < 100)")

Motivation

Delta Standalone can be used to create delta tables and adding constraints to tables is important aspect of it.

Further details

If this is not supported, can create a Feature request.

Willingness to contribute

The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?

• [ ] Yes. I can contribute this feature independently.
• [x] Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
• [ ] No. I cannot contribute this feature at this time.

[tdas]

As I have responded in the other issue #2984 standalone is not going to get any new functionality. We want to add all these new functionality in the Delta Kernel project.

Can you tell us a bit more about how you are using Standalone and why do you want to add constraints using standalone? Are you building a connector for an engine or are you just building your own application to directly operate on delta tables?

[JaySardhara]

Thanks for update @tdas. We are using delta-standalone currently to create delta tables in a non-spark way since we wanted to keep delta table creation times to minimal and delta-standalone offered best way since it just operates on writing metadata on parquet files. Hence in this process, we do have usecases where table creation would need constraints to be handled so that table is created with proper rules/validations.

Let me catch up on delta kernel offerings and see if that could still be a fit for why we were using standalone in 1st place.

[vkorukanti]

@JaySardhara Constraints are tricky to support in Standalone. In Kernel, we took into this account as part of the design for writes into the Delta table. The current constraints can contain any Apache Spark supported expression and it is not easy to support all these expressions in a non-Spark engine. In Kernel, we are starting with a few handful of expressions and expanding the supported expression as we get to know the usage of constraints. Let us know what kind of expressions are you planning to use within constraints. See the design decision here.