hadoop-crypto
hadoop-crypto copied to clipboard
EncryptedFileSystem does not support recursive delete
https://github.com/palantir/hadoop-crypto/blob/develop/hadoop-crypto/src/main/java/com/palantir/crypto2/hadoop/EncryptedFileSystem.java#L147
I may be missing some complexity, but it seems like recursive delete would just be a few more lines. Is there something I'm missing that would make this more complicated? Would a PR to support this be accepted?
Just realized (it's not in the README but) there's a StandaloneEncryptedFileSystem that already supports recursive delete. I don't really see why to use one or the other, other than that StandaloneEncryptedFileSystem has a kind of weird requirement to insert an e
at the start of the uris that it then removes.
We need to make KeyStorageStrategy
expose recursive delete and rename options in order for EncryptedFileSystem to support recursive delete and move. This is unlikely to happen any time soon, so the recommendation in the short term is either use StandaloneEncryptedFileSystem or use the EncryptedFileSystem with the FileKeyStorageStrategy and recursively delete or move on the underlying filesystem.
@ellisjoe is it still true that exposing recursive delete and rename options in KeyStorageStrategy
won't happen anytime soon?
@rahij we should talk about your need for recursive delete. Hasn't been a priority thus far.
Synced offline with Joe, we have a workaround for the internal library.