mikf
Change settings mid-download via script or exec postprocessor?
Is it possible to change settings with a bash script when something happens in a download for example, hitting 429 or 403 error codes, cookies not working anymore, etc in the middle of a download?
I run gallery-dl through a bash script that changes settings based on certain conditions based on which domain is being downloaded from for things gallery-dl can't do by itself.
This could also be useful to people within gallery-dl if it's done via a exec post processor instead of a wrapper script like mine.
Changing setting mid-download would maybe be possible via python post processor and metadata-extractor by modifying the extractor's internal state.
Doing this with an external script is currently impossible and would require gallery-dl to listen on a socket / implement some sort of server for another program to send "commands" to. Maybe even some sort of local API server like Hydrus. (Or maybe a signal handler that causes gallery-dl to re-read its config file, but that's not portable)
hitting 429 or 403 error codes, cookies not working anymore
Cursor support, i.e. being able to resume an extractor run from where it left off, might help here. I'll probably add something like this for Twitter before the next release, similar to Instagram.
based on which domain is being downloaded from for things gallery-dl can't do by itself.
Maybe we could add a (post processor) event or something to that extend that triggers for unsupported URLs, as they can currently only be written to a file?
+1 to this.
I'm finding Kemonoparty's DDosguard is checking cookies periodically. So I'm thinking about a wrapper that calls gallery-dl and watches for errors. After so many errors, it pauses the script and waits for input.
Open to other suggestions, of course.
There are many ways this can be useful to me. If it works with the yt-dlp extractor that might be especially useful, I'm not sure that is possible though