Multiple problems with S3 Backend documentation and usage

[eproxus]

File: themes/default/content/docs/intro/concepts/state.md

I am trying to use DigitalOcean Spaces as an S3-compatible backend. I encountered the following documentation problems:

1. It is not documented that the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY needs to be set.
2. It is not documented that AWS_REGION needs to be set to some bogus value which is apparently not used (otherwise an error occurs: MissingRegion: could not find region configuration)
3. When importing state exported from the Pulumi cloud storage, this error occurs:

   $ pulumi stack export --show-secrets --file infra.checkpoint.json
   $ pulumi logout
   $ pulumi login 's3://my-space?endpoint=fra1.digitaloceanspaces.com'
   $ pulumi stack import --file infra.checkpoint.json
   error: could not deserialize deployment: constructing secrets manager of type "service": could not find access token for https://api.pulumi.com, have you logged in?
   

4. After the stack import fails, Pulumi keeps asking for a passphares for every operation (even though the passphrase was empty):

   $ pulumi up
   Enter your passphrase to unlock config/secrets
       (set PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE to remember):
   

I realize some of these problems might be bugs in Pulumi itself. Please let me know how to handle that (report new issues etc.).

[lukehoban]

Issue (4) was addressed in https://github.com/pulumi/pulumi/issues/6974.

Issue (3) sounds like it is most likely by-design. Your original stack was using the Pulumi Service secrets provider, and then you exported that, but when you tried to import it into S3-compatible storage after logging out, you could no longer access that secrets provider. You most likely need to use https://www.pulumi.com/docs/reference/cli/pulumi_stack_change-secrets-provider/ to change the secrets provider before exporting, so that you can continue to use the stack entirely outside of the Pulumi Service. That said - we could be more helpful in the error message here - pointing to the likely need to change the secrets provider.

Issue (1) and partly (2) are indirectly addressed in the docs below. But we could definitely be more explicit on the common case configuration:

To configure credentials and authorize access, please see the AWS Session documentation. For additional configuration options, see AWS Setup. If you’re new to AWS S3, see the AWS documentation.

And (2) itself appears to be an oddity of Digital Ocean Spaces + AWS Go SDK - which is addressed in e.g. https://docs.digitalocean.com/products/spaces/resources/s3-sdk-examples/. We should add explicit examples here for some of the common alternative S3-compatible backends.

PRs welcome for improvements on any of these BTW!