yt-backup icon indicating copy to clipboard operation
yt-backup copied to clipboard

[Request] Size calculation of total downloaded on subsequent runs, rather than rclone size every time

Open sideloading opened this issue 5 years ago • 3 comments

Hi,

It would be great if instead of using rclone to generate the size of the directory on every run (which can take a while on large directories, and cause unnecessary spin-up of disks), and instead do this on first run (add to DB), then on subsequent runs it calculates the downloaded size from single runs.

Steps to make it clearer:

  1. First run: generate size of directory using rclone command -> added to DB
  2. Next run: see size of directory already generated -> skip rclone command 2.1 if files downloaded on this run, calculate size -> add to DB

And then you could have the optional statistic command to update the total directory size if necessary.

sideloading avatar Jun 11 '20 23:06 sideloading

I thought about the idea. In general it's a good idea. But what to do, if there users, which are adding videos to there archive with other tools?

w0d4 avatar Jun 16 '20 06:06 w0d4

Well, I've replaced my tool with this project, and it's working very well 😄

But they could use a generate_statistics command for their tools, if they wanted to.

sideloading avatar Jun 16 '20 09:06 sideloading

I ended up commenting out the following on L1820:

if mode == "run":
    verify_channels()
    get_video_infos()
    download_videos()
    #verify_offline_videos()
    #add_missing_channel_countries()
    #generate_statistics(True)

then running the following command seperately (e.g. once a day)

$ python yt-backup.py verify_offline_videos
$ python yt-backup.py generate_statistics --statistics archive_size,videos_monitored,videos_downloaded

An extra benefit was significantly reducing my API call as I was nearly reaching the 10,000 daily limit. Now down to ~1,000 avg with 998 channels.

image

sideloading avatar Nov 02 '23 03:11 sideloading