OptimizeRasters icon indicating copy to clipboard operation
OptimizeRasters copied to clipboard

Published 20 hours ago • verified publisher icon Esri

S3 Files >2GB Cause OverflowError Due to SSL

Open mbeaucha opened this issue 3 years ago • 2 comments

In python versions earlier than 3.10 the standard OpenSSL library version used only supported 32-bit reads. The result of this is that when trying to use OptimizeRasters on an S3 file that is > 2GB it fails with a non-intuitive error "signed integer greater than maximum value". The proposed code limits the amount of data that will be read in a single shot from S3 to avoid this issue. I have tested and verified this code against an image greater than 2GB.

mbeaucha avatar Dec 14 '21 22:12 mbeaucha

@Chamlika can you review the changes

abhiataero avatar Dec 19 '21 16:12 abhiataero

@Chamlika can you review the changes

Pull request not required, an updated fix has been added via #138. @abhiataero

Chamlika avatar Jan 25 '22 10:01 Chamlika