delta
delta copied to clipboard
Get delta table files
I checked the answer here: https://github.com/delta-io/delta/issues/623
but wondering whether the of inputFiles
is a reliable enough method of getting all the files? https://spark.apache.org/docs/3.1.3/api/python/reference/api/pyspark.sql.DataFrame.inputFiles.html
Docs are a bit vague on how reliable this is, we hosting our tables on S3