docker icon indicating copy to clipboard operation
docker copied to clipboard

Add S3 object store storage class option

Open aentwist opened this issue 1 year ago • 1 comments

User Story

When using S3-compatible object stores, choosing the storage class for objects is an important consideration. By tailoring the storage class to the application requirements there can be massive cost savings.

As a small-time user I want to be able to choose a cost-efficient storage class for light yet bursty usage (intelligent tiering). I do not want to mount additional files / add custom configurations to the base Docker image. I want to interact with this setting the same as with all other similar settings - by environment variable.

Proposed solution

Recently, this feature appears to have been implemented into nextcloud/server via https://github.com/nextcloud/server/pull/36075. The goal of this issue is to add it to the swath of environment variable settings available for the Docker image as described in the README.

To use an external S3 compatible object store as primary storage, set the following variables:
OBJECTSTORE_S3_HOST: The hostname of the object storage server
OBJECTSTORE_S3_BUCKET: The name of the bucket that Nextcloud should store the data in
OBJECTSTORE_S3_KEY: AWS style access key
OBJECTSTORE_S3_SECRET: AWS style secret access key
OBJECTSTORE_S3_PORT: The port that the object storage server is being served over
OBJECTSTORE_S3_SSL (default: true): Whether or not SSL/TLS should be used to communicate with object storage server
OBJECTSTORE_S3_REGION: The region that the S3 bucket resides in.
OBJECTSTORE_S3_USEPATH_STYLE (default: false): Not required for AWS S3
OBJECTSTORE_S3_LEGACYAUTH (default: false): Not required for AWS S3
OBJECTSTORE_S3_OBJECT_PREFIX (default: urn:oid:): Prefix to prepend to the fileid
OBJECTSTORE_S3_AUTOCREATE (default: true): Create the container if it does not exist

Check the Nextcloud documentation for more information.

Following the naming convention and current default storage class of STANDARD, let's go with

OBJECTSTORE_S3_STORAGE_CLASS (default: STANDARD): The storage class to apply to all stored objects

Alternative Considerations

  • How will this behave for S3 compatible stores not S3? Do they all support it? In any case, that is probably out of scope of this issue. We just need to interact with the setting that has already been implemented.

Notes

  • available from Nextcloud 26
  • currently undocumented

aentwist avatar Mar 03 '23 03:03 aentwist

workaround: get the updated s3 config file from the MR and mount it over the existing s3 config file at /usr/src/nextcloud/config/s3.config.php to support OBJECTSTORE_S3_STORAGE_CLASS. unfortunately nextcloud does not deep merge configs, so if you don't mount over it then it will be overridden by the existing s3.config.php, so you have to provide a replacement for the entire file.

aentwist avatar May 12 '23 04:05 aentwist

Fixed in #2138

joshtrichards avatar Apr 23 '24 22:04 joshtrichards