dremio-oss
dremio-oss copied to clipboard
fix: check for UseFileSplitsFromInputFormat annotation in hive plugins
When using custom input formats with Hive that are parquet based the Hudi Project introduced the custom annotation "UseFileSplitsFromInputFormat" to tell hive plugins to explicitly use the file splits from the InputFormat rather than calling it from MapredParquetInputFormat. This enhances the hive2 and hive3 plugins in dremio to respect this annotation and call getSplits on the inputformat when found in order to ensure proper handling and filtering of splits/parquet files with custom input formats.
LGTM