LANraragi
LANraragi copied to clipboard
Difegue
Limit connections when downloading from internet
When attempting to download with internet links (in my case specifically using the chaika) it seems like it is trying to make all the connections too quickly which then causes errors. If possible can you ad a way of limiting to x amount of attempts with y amount of wait time in between. Granted I'm not sure if this is something that should actually be part of the plugin or the downloader of lanraragi itself.
Auto-retries were added a while back in f299029bfcd3a6af79bb3aedecc4fe75deb3253e , although I suspect it can still fail if you queue too much at once.
In this case it was a list of 100 links, but I have done less than that in the past with the same results
Currently Chaika is erroring out its downloads if i queue more than 3 links at once. Feel a rate limiter is needed at this point atleast for that site.
A year ago or atleast some months i could dump hundreds of download links from chaika without erroring out at all, now it errors out anything over 3 links at once.
Also if you try quite a few at once now you get a temp ban on your IP which happened with me when trying to trigger 200 urls.
Weird the site does this when you are able to generate hundreds of download links with a single button. The site is clearly made to be able to queue these up by the hundreds. Maybe its an user-agent issue along with the rate limit, forcing it to take the shape of a normal browser would make sense.
👋 Historically rate limiting was integrated into the plugins themselves, but that's inefficient and doesn't work if you queue multiple DLs with the same plugin.
There should probably be a way for a plugin run to return a status saying it's been rate limited alongside a waiting period, Which'd then put all runs for that plugin on pause until the waiting period ends.