Enable support for Google object storage credential file to be mounted on replication source/destination pods

[shubham-pampattiwar]

Describe the feature you'd like to have. Support for mounting GCS credential json file

What is the value to the end user? (why is it a priority?) We are currently using volsync with OADP Operator to move CSI snapshots to object storage. We were able to integrate volsync restic datamover for AWS S3 and Azure blob storage but are blocked on Volsyn restic with GCS.

How will we know we have a good solution? (acceptance criteria)

Can you have a way to mount the GCS credentials json via a volume mounted on replication source/destination pods, so that the file path can be passed to the restic env var GOOGLE_APPLICATION_CREDENTIALS for GCS to work with restic datamover ?

Thank you in advance !!

Also, Awesome controller guys !!

[JohnStrunk]

Since this is credentials, I assume the json file would be in a Secret, not a PVC?

[shubham-pampattiwar]

Basically we want a Volume on replication source pod whose VolumeSource would be a secret.

[JohnStrunk]

/assign

[JohnStrunk]

@shubham-pampattiwar (and @tesshuflower) I'm working through the design of this item, and I'd like your opinion...

Design 1

• Add a field to .spec.restic like credentialSecretName which then has its contents projected into /credentials in the container.
• To use, you'd
  • set GOOGLE_APPLICATION_CREDENTIALS to something like /credentials/secret-key.json
  • Create a new Secret having a key of secret-key.json and put the name of the Secret into credentialSecretName

Design 2

• Don't add an additional Secret
• Instead of passing a filename via GOOGLE_APPLICATION_CREDENTIALS, just embed the credential file itself as that key.
• The mover code would then internally do the "design 1" steps, mounting just that key as a file and setting the restic env var.

---

I like design 2 since there's less to get wrong w/ the configuration (can't get the path wrong; don't need to understand the volume mapping). The operator/mover are responsible for getting the file paths correct. The negative is that creating the restic secret is slightly more complicated, and it's no longer an exact 1:1 mapping between Secret "keys" and restic environment variables.

Do you see anything in design 2 that would make it difficult to use?

[tesshuflower]

From the perspective of someone not that familiar with setting up for restic, I would prefer Design 2 myself. It seems simpler from a setup point of view to keep my 1 restic secret with everything there, and not deal with mappings to mount points and additional entries to set in the spec.

That said, I'm not sure if users who are used to setting GOOGLE_APPLICATION_CREDENTIALS will find it confusing as now instead of a file name they are supposed to know to put the file contents as the key value in the secret? We could also introduce a new var, something like GOOGLE_APPLICATION_CREDENTIALS_FILECONTENTS - but not sure that really makes it much easier. Either way we'll just have to document it.

This is the only var so far that works like this correct? (i.e. env var needs to point to a path to a file rather than it directly containing a value to be used)

One more suggestion (maybe only worth it if we think there will be more vars that use this approach):

• Do something special for keys in the restic secret that start with credentials_
• user can use key credentials_secret-key.json with value being the file contents in their restic secret
• We would then mount this file in the container at /credentials/secret-key.json
• User can set GOOGLE_APPLICATION_CREDENTIALS to /credentials/secret-key.json

[shubham-pampattiwar]

@JohnStrunk I am good with design 2 as well, simple and straight forward. But yeah I kind of agree with @tesshuflower that users might be used to setting a file name.