imgbrd-grabber
imgbrd-grabber copied to clipboard
Fix Batch Downloader Counters
Bug description
Overview
The batch downloader has serious issues counting the number of total downloads and number of downloads already done.
Look at this above picture with numbers which never materialized.
- The above batch completed at ~44k images even though it stated ~86k images as total. At the start of batch, this number was near 185k.
- In total about ~15k images were actually downloaded
- Now usually it shows some completely different summary which numbers don't match with even this counted number.
- If there were any images with error and the retry prompt is shown, it shows me the summary for those error images and forgets the numbers for the actual batch.
- Canceling the operation also forgets to show the status of downloads till now.
I cannot know when it will end nor what the progress is or how many it actually downloaded. It does seem that it excludes random amount of images from the 1k items batches due to some filter and then ignores some (but counts them as done) while downloading the rest.
Typical Search Pattern
I have organized highly specific batches that I run every month and each successive batch excludes the previous main tag, for example the below search string is the last of all for gelbooru: (bold are blacklisted tags)
Filters:
height:>=1000 width:>=1000 score:>=10 -solo -large_breasts -smile -nude -original -pussy -swimsuit -sweat -open_clothes -tongue -1boy -sex -pussy_juice -cum -dress -tears -bondage -fellatio -rape -rating:safe -animated -yaoi -furry -male_only -futanari -white_background -shota -loli
Post Filters:
age:..6y -ext:swf -ext:gif -grabber:inMd5List
Expected behavior
The counters must follow one flow and be correct to the last digit. Current downloaded images and images left to be downloaded is more important of a metric for us than skipped images.
The completion summary must show us the summary of the whole batch including the error images and not forget show it at all in favor of showing summary of error images retry.
Cancellation must also show the downloaded statuses.
Context
- Settings and Blacklist: Grabber.zip
- OS: Windows 10 x64 2004
- Grabber version: Nightly
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If this issue is about a bug that still happens in the latest version, or a suggestion that is still relevant, feel free to comment on it and the maintainers will have another look, they might have missed it!
Thank you!