delta
delta copied to clipboard
Use IndexedFile for DeltaSource admission control instead of FileAction
Which Delta project/connector is this regarding?
- [X] Spark
- [ ] Standalone
- [ ] Flink
- [ ] Kernel
- [ ] Other (fill in here)
Description
This is a small refactoring. The DeltaSource has admission control (e.g. maxFilesPerTrigger), and the code is based on FileAction objects. However, all the calling code really has IndexedFile objects, and has to do manual conversions to deal with the fact that not all IndexedFile objects have file actions. This PR simplifies this by making the admission control based on IndexedFile.
How was this patch tested?
Existing unit tests.
Does this PR introduce any user-facing changes?
No.