goofys icon indicating copy to clipboard operation
goofys copied to clipboard

Stop file read after passing the Unencrypted length

Open tomekit opened this issue 1 year ago • 1 comments

Is it possible to "stop" the file read after reading the: "Unencrypted length" of bytes, which will be usually less than the encrypted length as presented in the "ls" / stat.

There are many client-side AES-256 implementations. The S3 stored binary is rounded up to the block size, whereas the real content will be usually shorter. The info about unencrypted length is often stored in the metadata in e.g. x-amz-unencrypted-content-length and is available after headObject or getObject. Unencrypted length isn't available via standard listing methods e.g. listObjectsV2.

I am experimenting with Goofys and modified the: internal/backend_s3.go:GetBlob() so it can decrypt the content, however if I don't pad the last block (so it matches the "stat" size) the GetBlob seem to be called again (until it returns the amount of data to satisfy the size advertised in the ls)

I don't know the Goofys/FUSE and file system internals and I am not sure who requests (during file read process) file contents up to the last byte. Would there be possibility to make Goofys/FUSE/filesystem aware during the execution (technically after first getObject response received with metadata) that file length is actually different then the one reported to ls, so we can return the file exactly as it is, instead of trying some padding tricks? Perhaps there is some EOF character I could return?

Sorry for my ignorance, but file systems and Go are entirely new things to me from the dev perspective.

tomekit avatar Nov 20 '22 17:11 tomekit