sprout3 icon indicating copy to clipboard operation
sprout3 copied to clipboard

Feat/files s3 backend

Open jamiemonksuk opened this issue 6 months ago • 0 comments

Reopening this to look at the implications and maybe reconsider

Some things I want to tidy up:

  • Migration tool (I'm changing it to allow a copy of files before the file system is swapped, otherwise dice will take days to migrate)
  • File edit/save in admin (too slow, might need to ship some things off to workers)
  • More testing and real world usage in dice
  • Proper final review (needs a good sense check with a cleared head)

I've had to do a lot of work and make a few compromises to get the site to load in a reasonable amount of time when using s3 (or other remote back end). This includes things such as making local copies of files before working on them, splitting default resizing up into instant (for things admin will need when the page reloads) and worker based (for front end things like small, medium, etc.

I've also added heavy caching layers for things like filesize and imagesize which otherwise require pulling an entire file down to inspect.

I think this is working pretty well now.

DB MIGRATION TOOL

We need to be able to split up the migration, so we can copy large numbers of files without borking a site in between. I've had a red hot go at this, by adding a tool to prepare / finalise / both at once.

This uses additional *migrated cols on the record. This will need a proper good test

image

image

image

jamiemonksuk avatar Aug 16 '24 04:08 jamiemonksuk