hadoop-crypto icon indicating copy to clipboard operation
hadoop-crypto copied to clipboard

EncryptedFileSystem does not support recursive delete

Open TheBeruriahIncident opened this issue 6 years ago • 5 comments

https://github.com/palantir/hadoop-crypto/blob/develop/hadoop-crypto/src/main/java/com/palantir/crypto2/hadoop/EncryptedFileSystem.java#L147

I may be missing some complexity, but it seems like recursive delete would just be a few more lines. Is there something I'm missing that would make this more complicated? Would a PR to support this be accepted?

TheBeruriahIncident avatar Apr 19 '18 02:04 TheBeruriahIncident

Just realized (it's not in the README but) there's a StandaloneEncryptedFileSystem that already supports recursive delete. I don't really see why to use one or the other, other than that StandaloneEncryptedFileSystem has a kind of weird requirement to insert an e at the start of the uris that it then removes.

TheBeruriahIncident avatar Apr 19 '18 15:04 TheBeruriahIncident

We need to make KeyStorageStrategy expose recursive delete and rename options in order for EncryptedFileSystem to support recursive delete and move. This is unlikely to happen any time soon, so the recommendation in the short term is either use StandaloneEncryptedFileSystem or use the EncryptedFileSystem with the FileKeyStorageStrategy and recursively delete or move on the underlying filesystem.

ellisjoe avatar May 09 '18 10:05 ellisjoe

@ellisjoe is it still true that exposing recursive delete and rename options in KeyStorageStrategy won't happen anytime soon?

rahij avatar Apr 25 '19 11:04 rahij

@rahij we should talk about your need for recursive delete. Hasn't been a priority thus far.

ellisjoe avatar Apr 29 '19 10:04 ellisjoe

Synced offline with Joe, we have a workaround for the internal library.

rahij avatar Apr 29 '19 10:04 rahij