caching
caching copied to clipboard
nette
SQLiteStorage does not automatically clean() expired cached items
Nette Cache FileStorage runs clean() with a 0.001 probability - a sane and sensible default, works great.
However the other provided storage methods, such as SQLiteStorage and MemcachedStorage, do NOT automatically run clean(), instead relying on the developer to add their own mechanism to periodically delete expired entries.
This makes the behaviour of the different storage methods inconsistent.
In addition, the need for developers to add their own mechanism is not mentioned anywhere in the documentation.
OK, I wrangled a fix and put together a pull request. The actual code is only a few lines, the test suite took me way longer lol
Some background here - I use Nette Cache to cache about 3-5 gigabytes of HTML strings (website search results). I used FileStorage for years, but at a certain number of files it starts periodically throwing errors which I figured were related to the sheer number of files that needed to be parsed during certain operations (there were hundreds of thousands of files over >100 folders).
This issue was fixed by switching to SQLiteStorage, which doesn't have any problems when the cache gets large. However, without the expired entries being clean()'d automatically the DB file grew over time to almost 40 GB, and the space on my VPS started running low. I manually deleted the expired entries in the DB, but this doesn't actually reduce the disk size of the DB file.
The solution to the filesize issue is to do a VACUUM() command in SQLite, but this requires enough drive space for two copies of the DB file, which I definitely did NOT have. I eventually deleted the whole file and started afresh. A hassle, to say the least!
Adding an automatic clean() to SQLiteStorage will prevent the DB getting out of control for other users.