YoutubeDL-Material icon indicating copy to clipboard operation
YoutubeDL-Material copied to clipboard

[FEATURE] Warn users when local DB is too large

Open Tzahi12345 opened this issue 1 year ago • 1 comments

Is your feature request related to a problem? Please describe. We have one example (#894) of issues that happen when the local DB JSON gets too large, which was at a point before it reached 540 MB.

Describe the solution you'd like At server startup, before the DB is read, we should check if the user is using local DB or not. If so, then if the file is larger than say, 400 MB, we should issue a warning to have the user transfer to MongoDB. This can also be based on performance issues that inevitably occur when this file becomes too large, @GlassedSilver could probably give more context into when that happens.

Tzahi12345 avatar May 07 '23 03:05 Tzahi12345

Yeah, you'll notice when that happens fairly quick, everything becomes ugly. I'm advocating for a much lower limit, because to be fair, unless you're only doing one-off downloads, the second you have any kind of automated ingress like subscriptions you're already kinda starting to walk the path of continuous growth. Since it's a warning and not a halt of operation it's probably wise to set the warning at 100 MB.

I consider operating without using the dedicated db backend to be a trial operation convenience at this point.

This is even more true when you consider that unsafe shutdowns or similar can completely wipe the contents of the database file, something that proper db apps have much less of a problem handling.

GlassedSilver avatar May 07 '23 04:05 GlassedSilver