delta
delta copied to clipboard
[UNIFORM] Remove the `setSchema` patch
This patch aims to align the field-ID assignment between uniform and Iceberg. This way we can use the public Iceberg APIs to evolve the schema, instead of enforcing the schema through a private API.
The difference is that Iceberg does not have true pre-order traversal, but when it encounters a struct, it reserves an ID for each field in the struct before continuing the traversal.
Iceberg:
1: struct
2: struct
4: long
5: timestamp
3: int
Delta:
1: struct
2: struct
3: long
4: timestamp
5: int
This PR changes Delta's assignment behavior, so we don't need the patch anymore.
Which Delta project/connector is this regarding?
- [x] Spark
- [x] Standalone
- [ ] Flink
- [ ] Kernel
- [ ] Other (fill in here)
Description
How was this patch tested?
Does this PR introduce any user-facing changes?
cc @amogh-jahagirdar