delta icon indicating copy to clipboard operation
delta copied to clipboard

[Spark] Test type widening compatibility with other Delta features

Open johanl-db opened this issue 9 months ago • 1 comments

Description

Additional tests covering type widening and:

  • Reading CDF
  • Column mapping
  • Time travel
  • RESTORE
  • CLONE

How was this patch tested?

Test only

johanl-db avatar May 06 '24 12:05 johanl-db

Added a few suggestions for additional tests. Do we already have tests for constraints and generated columns. AFAIK we disallow type changes on generated columns, but constraints might be interesting as it's possible to create a constraint after changing the type.

There are tests already for generated columns and constraints, added as part of https://github.com/delta-io/delta/pull/2881

I added a test here to cover constraints + RESTORE which is an interesting case:

  1. Change column a type from byte to int
  2. Add a CHECK constraint on column a.
  3. RESTORE table to before the type change.

The constraint was added on type int and the column now has type byte. This does work though because CHECK constraints are part of the table metadata and it gets removed during RESTORE.

johanl-db avatar May 17 '24 15:05 johanl-db