press
press copied to clipboard
refactor(s3): Don't hardcode endpoint and region
Don't hardcode endpoint and region for S3 buckets
There were many references to the ap-south-1
region and a global implicit reference to AWS. This PR contains refactors of the S3 bucket systems for "offsite backups", "dashboard uploads" and "Backup Bucket doctype" buckets, to avoid vendor lock-in.
I tried my best to make it backwards compatible with what I assume is the configuration on Frappe Cloud, but I can't check.
Details:
- Added region and endpoint fields in Press Settings for the "offsite backups" and "dashboard uploads" buckets
- Fields are:
offsite_backups_endpoint
,remote_uploads_region
,remote_uploads_endpoint
- Fields are:
- Dropped unused fields:
offsite_backups_provider
(replaced by endpoint),data_40
(typo probably) - There might be some agent stuff left that is not handled correctly (e.g. retrieve the ENDPOINT_URL param here)
- I did not add unit tests
~~cc: @gavindsouza :wave:~~ re: https://github.com/frappe/press/pull/19 https://github.com/frappe/press/pull/22
For the Frappe team
- There was a hardcoded
uploads.frappe.cloud
URL somewhere, I just kept it unchanged, but the refactor won't be truly finished until a migration path can be found for you. - I'm not so sure about the other buckets (Backup Bucket doctype), they already had the Endpoint and Region fields, but these were not used everywhere (e.g. not in delete_s3_file)
Typical configuration values for the fields
AWS S3 (Fallback/Legacy)
- Region: leave empty for
ap-south-1
- Endpoint: leave empty for automatic endpoint generation by boto3 = AWS S3 (I believe
https://s3.amazonaws.com
)
AWS S3
- Region:
ap-south-1
- Endpoint:
https://s3.ap-south-1.amazonaws.com
(possibly a custom DNS name such asuploads.example.com
)
OVH
- Region:
de
- Endpoint:
https://storage.de.cloud.ovh.net