stashapp
[WIP] Add filesystem based blob storage
This is a replacement PR for #2491.
Our existing blob storage system is a number of tables containing images - scene_covers, performers_image, etc. Additionally, scene covers are stored both in the database and in the filesystem, in the generated/screenshots directory.
This PR consolidates all of these blobs into one location, either a single table in the database, or in the filesystem.
Adds a new table blobs which is a pair of a string checksum value and optionally the blob data. When a blob is added to the system, an MD5 checksum is generated, and this key is used to insert the data into the blobs table. If the filesystem storage system is used, then the blob data is instead stored in <blobs>/xx/xx/checksum, where <blobs> is configurable, and xx refers to the first four letters of the checksum, used to split the files up. The referring objects now have columns for each of the blobs and the old tables are dropped - eg movies now has front_image_blob and back_image_blob with foreign keys to the checksum column.
Migrates all image implementations to this system, including scene covers. Scene covers will be stored in this system only, and the screenshot files will not be used unless a cover can't be found in the database. They will not be regenerated.
The cover generation is now split out into a generate task. This means that covers may optionally not generated during scanning, and performed instead in the generate task.
Incidentally, there is now an Is Missing Cover scene filter criterion.
The setup wizard is changed to accept a blobs location, or to give no location and use the database storage system. An option to change between systems is in the System settings.
TODO: The schema migration will not migrate covers in the generated/screenshots directory. This will be provided as a post-migration step. This migration will copy the images in the screenshots directory into the blob system - either into the database or the blobs filesystem - ignoring those which already have a cover blob. It will optionally delete the screenshot files as they are migrated, to save on disk space.
TODO: Migration tasks will be implemented to migrate between systems. This will be a mandatory task that occurs when changing between the database and filesystem. The migration will involve reading the blob files in the filesystem and setting them in the database, or vice versa.
Existing systems will migrate their database blobs automatically, and the blob storage system will be set to database by default if there are any image blobs in the system, otherwise filesystem is the default. filesystem is also the suggested default for new systems.
This is not quite finished as denoted by the TODOs above, but the core of it is ready for testing and feedback on the concept. The only significant drawback to the current approach is that one can no longer preview the files in Windows since the stored files do not have any extension.
Resolves #2303 Resolves #2271
