Implement inspect data API v2

[vadmeste]

Description

The old inspect format is still supported

The new format supports data encryption with a public key, that will be received from madmin-go/mc.

The real data encryption key is still random, and it will be encrypted with the provided RSA public key. The encrypted key will be shipped in the archive as key.enc

Motivation and Context

Encrypt backend data in inspect archive with a public key

How to test this PR?

mc support inspect myminio/testbucket/format.json

Types of changes

• [x] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Optimization (provides speedup with no functional changes)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• [ ] Fixes a regression (If yes, please add commit-id or PR # here)
• [ ] Documentation updated
• [ ] Unit tests added/updated

[harshavardhana]

Please rebase the PR @vadmeste - PTAL @aead @klauspost

[vadmeste]

Instead of recursive ZIP (which is annoying to unpack) we can use a format similar to v1

We wanted to have encrypted data + clear text files inside the zip file, the current way seems to be the easiest way

[vadmeste]

We wanted to have encrypted data + clear text files inside the zip file, the current way seems to be the easiest way

Well, unless we define our own format for inspect data, and it would be a file which contains our clear text data in the beginning, then the encrypted zip file

[klauspost]

Well, unless we define our own format for inspect data, and it would be a file which contains our clear text data in the beginning, then the encrypted zip file

We are defining it already. Adding a zip-inside a zip doesn't really help anything, since the data will be unreadable anyway.

If you look through the proposed change, the only difference is that we add the encrypted key before the encrypted data instead of inside a file in the zip file.

There is no need for the double wrapping. It makes decryption needlessly complicated, since you need to store a temporary file to access the content.

[harshavardhana]

madmin-go v1.7.0 is released with @klauspost changes - cc @vadmeste this PR can move forward now.

[harshavardhana]

PTAL @aead @klauspost

[minio-trusted]

Mint Automation

Test	Result
mint-large-bucket.sh	:heavy_check_mark:
mint-fs.sh	:heavy_check_mark:
mint-erasure.sh	:heavy_check_mark:
mint-dist-erasure.sh	:heavy_check_mark:
mint-compress-encrypt-dist-erasure.sh	:heavy_check_mark:
mint-pools.sh	:heavy_check_mark:
Deleting image on docker hub
Deleting image locally