Ljzd-PRO
Incomplete Corrupted Images
When downloading with default settings sometime photo images will download but be halfway complete. So when you view the image you see half of the actual image while the other half of it is black. In the debug console only thing that pops up sometimes when this occurs is this
"PermissionError(13, 'The process cannot access the file because it is being used by another process')"
So is it related to concurrent downloads? Do I need to lower the default 10 or do i need to mess with the chunk and buffer size? Can HDD be affecting it and should only do it on SSD?
Also with many folders theres no way to delete each corrupted image easily is there no SHA/Checksum feature to have it check and redownload?
Specs Windows 10 CPU 9950x3D RAM 128GB
This is not related to concurrent downloads. For each file, there will only be one download thread. The term "concurrent downloads" refers to the concurrency in multi-file download tasks. While this issue might have occurred in the past, the probability was relatively low, and I haven't encountered it recently.
This is unrelated to the type of hard drive. Additionally, the program does not employ multi-process technology. The concurrency is implemented asynchronously within a single process and a single thread.
Hash verification can indeed be added.
For now, you can check if the same files consistently encounter errors. Try modifying directory permissions, changing the download directory, and test both the download-post and sync-creator commands separately to see if the errors persist in both scenarios.
I know the reason for this issue - it's because another post with the same name and timestamp was downloaded together. This is a Kemono issue where they fetched the same post twice.
Hello,
So when I first commented this I thought it was a download connection issue but a few days later I noticed this issue only happens if a post has duplicate links/pictures within a post. So if two link paths are the same KToolBox will try to download both at the same time and then thats when that error occurs and a file corruption happens when one of the two finishes first. So what Kochisa says also applies. This issue doesn't happen with anything else besides what me and Kochisa describe. I tried another tool(cyberdrop-dl) to see if it occurs with it also but it doesn't so they must have some type of code that takes that into consideration.
Was going to make a new issue post, but this one is related and still open, so hope op and Ljzd dont mind!
Want to suggest 2 things:
- add an option to delete failed/corrupted files at the end of a run;
- add a "new rerun" those deleted failed/corrupted files, so it can complete all (if the problem was momentary).
