Daniel Weeks
Daniel Weeks
@wypoon I was able to get this up and running based on your branch, so i think we can make it work. As for the comparison to the Spark distribution,...
@wypoon I appreciate the effort put into getting this working, but I'm concerned about the approach, complexity, and unintended impacts this approach may have. I see the goal here as...
@wypoon I was out for a bit, but back looking at this. I'm fine with updating to use Hive 4 as the primary target and test against the other versions....
Hey @wypoon, I spent some time working on getting the test only approach working, but ran into some other issues that I'm worried are even bigger problems with our Hive...
I think we should take this back to the dev list and discuss since we've gone from just testing to producing new artifacts. This could potentially affect other area of...
@steveloughran This isn't about bulk deletes (which S3FileIO does support). The issue is how to properly scale the identification of orphaned files, which is function of the procedure, not the...
Thanks @bryanck and @fqaiser94. It's really great to get this one in.
@flyrain I'm a little confused, how can the REST Server not have access to the files? Currently the server needs access to at least the metadata files. Are you considering...
Just adding the original doc for reference: https://docs.google.com/document/d/1UxXifU8iqP_byaW4E2RuKZx1nobxmAvc5urVcWas1B8/edit#heading=h.6sa1rpsxiuke
> Do we still pursue this? Yes, @amogh-jahagirdar is still looking at this and we continue to see use cases where this is necessary.