Restore of database backup from S3 storage fails

[MarkDoggen]

Restore of a compressed, encrypted database backup fails. Media restore (also compressed and encrypted) works fine though.

Media:

python manage.py mediarestore -z -e --passphrase pass
Restoring backup for media files
Finding latest backup
Restoring: media-2018-10-08-223640.tar.gz.gpg
Are you sure you want to continue? [Y/n] Y

Database:

python manage.py dbrestore -z -c --passphrase pass 
Finding latest backup
Restoring backup for database 'default' and server 'None'
Restoring: default-2018-10-08-221305.psql.gz.gpg
Restore tempfile created: 0 B
Are you sure you want to continue? [Y/n]

I thought it might be the size of the backup, so I started with truncated tables but it also shows the same behavior (Restore tempfile created: 0 B)

Does anyone have any clue what might be the issue here?

[ZuluPro]

@MarkDoggen What's your backup size from the storage ?

[MarkDoggen]

It is 1.2 GB (compressed and encrypted)

[bjassael]

Hello, I had the same issue. Looking at the code, the only difference from applying both options together (decrypt and uncompress) is that the function uncompress_file receives a SpooledTemporaryFile instead of a normal file. Then the following line is responsible of uncompressing the file: zipfile = gzip.GzipFile(fileobj=inputfile, mode="rb") Probably GzipFile is not working with SpooledTemporaryFile. A possible solution is to first encrypt the file and then compress it. I check the size of the encrypt-compressed file after changing the order and is even smaller than compressing-encrypting.

[galt]

You can't compress an encrypted file. Compress first, then encrypt.

[bjassael]

@galt, I tested encrypting first and worked fine. Even, the encrypted-compress backup is smaller than the other way around.

Here a column that recommends encryption before compression for security: https://blog.appcanary.com/2016/encrypt-or-compress.html

@ZuluPro ZuluPro it's possible to commit this change?

[galt]

Your results seem miraculous. Please double-check them. Can you decompress and decrypt both of the files and recover the complete and correct original input? Perhaps you have labeled them backwards?