Refactor backend storage, store just the diffs in a DB

[dgtlmoon]

This could have some really nice impacts

• Reduced volume size
• Can find which other pages had the same differences and ignore them/know about it
• SQLite could be good for this
• 'Backup' mechanism may need a refactor if you want the original snapshots back again
• Why not use GIT perhaps? just an idea

[immanuelfodor]

Please don't use SQLite, instead invest a bit more into using an RDBMS over TCP/IP (like MariaDB, PostgreSQL). SQLite is a PITA when using Kubernetes with more than one node with NFS (most homelabs).

[DieselTech]

And on the other side of the coin, there are plenty of setups that don't have a dedicated database instance running and having to stand one up just for 1 software package isn't worth it. I have over a dozen services running in a homelab environment and have yet to need one. Most software packages do fine with SQLite and it makes it easier to back things up. Especially in a docker environment where your backups can just be a folder grab of your container datastore.

Kubernetes with more than one node with NFS (most homelabs).

I would argue "most homelabs" are single unit setups. Something like a rpi / old PC or many of the NAS units now that come with docker abilities.

[immanuelfodor]

I must say that all *arr and media apps (Radarr, Sonarr, Plex, Jellyfin, etc) are a pain to host in Kubernetes because all of them use SQLite. They must be fixed to a node and can only work with hostPath storage, so when a specific node goes down where their volume is pinned, the app goes with it.

Using an ORM that can work with multiple DB engines (SQLite, PG, My/MariaDB) can be a solution for both worlds :)