cms
cms copied to clipboard
statamic
[5.x] Enable background re-caching of static cache
This PR enables background re-caching of the static cache through a new static_caching.background_recache variable (or STATAMIC_BACKGROUND_RECACHE .env value).
When set to true the static cache will re-cache the page without invalidating the previous cache, so at every point a static file will exist and end users will not encounter slow loading pages.
Invalidation continues to occur as it currently does when content is deleted, or when this variable is set to false (the default).
Do I understand correctly, that this will serve the old cached file until the new cached file is ready? And then it gets rid of the old cached file? Or what exactly is the purpose of this? Sounds interesting for sure.
Also, from a configuration point of view, it might be easier to just have the config to be a boolean and then assign a random string automatically if the config is true.
this will serve the old cached file until the new cached file is ready? And then it gets rid of the old cached file?
When the new cached file is ready, it'll just stomp over the top of the old one.
it might be easier to just have the config to be a boolean and then assign a random string automatically if the config is true.
We need to know the value ahead of time, so it can't just be a randomly changing string.
@aerni the idea of it being a fixed string was to provide some sort of security - that someone malicious couldn’t just constantly force your site to recache.
Personally, I'd be a little intimidated by the instruction to "add a unique string here that should not be easy to guess". Like, what is "easy to guess"? Couldn't we just use the APP_KEY as the unique string?
When the new cached file is ready, it'll just stomp over the top of the old one.
By "stomp over" I suppose you mean the old file will just not be served anymore? Won't this result in a lot of abandoned files pretty fast? Should there maybe be some sort of cleanup?
Maybe we could make it a Boolean config and the token could be a Crypt or Sha of the url. The app key is used in the Crypt so it wouldn’t be game-able.
By "stomp over" I suppose you mean the old file will just not be served anymore? Won't this result in a lot of abandoned files pretty fast? Should there maybe be some sort of cleanup?
it literally replaces the same file. So no clean up will be needed.
Maybe we could make it a Boolean config and the token could be a Crypt or Sha of the url. The app key is used in the Crypt so it wouldn’t be game-able.
Oh yeah, makes sense. It's early morning here 😄
Love it, thanks! Curious as to why you decided to make this opt-in, though. Feels like everyone who uses static caching would want to enable this feature. Or is there any potential downside to it?
this will serve the old cached file until the new cached file is ready? And then it gets rid of the old cached file?
When the new cached file is ready, it'll just stomp over the top of the old one.
An edge case idea: In theory, isn't file access concurrency a concern here? What if file is being read and locked and you try to overwrite it? What if being written to and server tries to read it? I realize this is purely theoretic and performance and reliability impact is probably negligible, but ....
@Krzemo Are those things not already a potential issue with the existing implementation? My understanding is modern filesystems will lock the file during writing, so any read operations will wait until the file is unlocked, so it shouldn't be an issue.
@ryanmitchell Im not worried about OS not doing its job.
Maybe asking different questions will help explain what I mean (I feel like I should run tests myself first - will do next week if still needed):
- Does file get locked when HTML generation starts and freed when done or HTML is ready first and file gets locked for a brief moment only when it is being written? Can a file be locked because of first option for i.e. 1 minute+?
- Does locking a file for writing make it unreadable? Can it end up in the same situation like with no background refresh where a user have to wait for file generation to finish not because there is no static file, but because file is locked for reading?
I understand now. The file is only locked during the writing not during the regeneration. So it should be a fraction of a second.