Huy Nguyen
Huy Nguyen
Kind of cheating but a naive solution is to use pandas json_normalized to parse the json and then convert the resulting pandas df into Spark. The logic seems a bit...
Sadly the transaction log information seems to only be exposed in the Scala version, not Python one :( https://books.japila.pl/delta-lake-internals/DeltaLog/ If we wanna do this in pyspark, we would have to...
Yup, levi might be a more straightforward place for this feature. I'll raise an isse and look into implementing it