gallery-dl
gallery-dl copied to clipboard
mikf
[Feature Request] Extractor for Deviantart gallery search
Although a extractor for user galleries is already present, it doesn't works if you try to download the results of a search did in a given gallery. Example: gallery-dl https://www.deviantart.com/otakurenders-service/gallery?q=isuzu [gallery-dl][error] No suitable extractor found for 'https://www.deviantart.com/otakurenders-service/gallery?q=isuzu'
Can't promise I fixed it, but I tried, 0135am no less.
Still not working, I got same error :( I already upgraded gallery-dl to the latest version (1.18.1).
gallery-dl, like youtube-dl, is manually published, and evidently mikf has to manually approve a PR before the code starts compiling, so you need to wait for that.
Once he approves (and preferably wait for the compilation to succeed) you need to download the whole source code and that way test it by running the __main__.py with your params and flags.
With https://github.com/mikf/gallery-dl/pull/1696 it would accept those gallery search URLs, but it wouldn't do the requested search and just download the same as /gallery/all.
Actually implementing this feature is also more or less impossible as long as gallery-dl uses the official OAuth API, simply because it doesn't provide that functionality. gallery-dl could theoretically go through all possible results and try to manually apply a search filter, or we start using the Eclipse API (https://github.com/mikf/gallery-dl/tree/deviantart-rewrite)
Me approving the PR wouldn't compile anything, by the way. It'd only run the tests from https://github.com/mikf/gallery-dl/blob/master/.github/workflows/tests.yml
With #1696 it would accept those gallery search URLs, but it wouldn't do the requested search and just download the same as
/gallery/all.
Ya, I figured that's the price to pay since DA's search facility is 99% useless since Eclipse.
Actually implementing this feature is also more or less impossible as long as gallery-dl uses the official OAuth API, simply because it doesn't provide that functionality. gallery-dl could theoretically go through all possible results and try to manually apply a search filter, or we start using the Eclipse API (https://github.com/mikf/gallery-dl/tree/deviantart-rewrite)
Huh. Didn't know there was another API.
Me approving the PR wouldn't compile anything, by the way. It'd only run the tests from https://github.com/mikf/gallery-dl/blob/master/.github/workflows/tests.yml
Ah ok. Well, as far as seeing whether the PR even works, the result is the same.
Committed a fix for the tests, I haven't quite learnt how to use GitHub to show that here, but it's visible in the PR.
I linked to a regex101 piece where all the tests succeed fine, and I tried to DL using OP's link, which worked perfectly fine too.
I'm just gonna admit defeat so that someone with better Git understanding (and the motivation) can get it through. I think you can see why I had a strong dislike for Git from the maintaining standpoint.
There are some other fringe dA cases I'm curious about, like what about outside of users just exploring tags?
There is actually a bit of a loophole. In the search box at the very top of the page, enter the following (replacing the search text and username, of course):
insert search text here username:insertusernamehere
This means that for the example in the first post, the correct URL to pass to gallery-dl would be:
https://www.deviantart.com/search?q=isuzu+username%3Aotakurenders
The only problem is that it fails to work with users with a - in their username. With the example in the original post, stripping everything that comes after the - (leaving only "otakurenders") is still precise enough to return the appropriate results. Unfortunately, this is not always realistic: for example, a search for hypothetical user "the-artist" would return everything with the search text by all accounts with either "the" or "artist" in the username.
There is actually a bit of a loophole. In the search box at the very top of the page, enter the following (replacing the search text and username, of course):
insert search text here username:insertusernamehereThis means that for the example in the first post, the correct URL to pass to gallery-dl would be:
https://www.deviantart.com/search?q=isuzu+username%3AotakurendersThe only problem is that it fails to work with users with a - in their username. With the example in the original post, stripping everything that comes after the - (leaving only "otakurenders") is still precise enough to return the appropriate results. Unfortunately, this is not always realistic: for example, a search for hypothetical user "the-artist" would return everything with the search text by all accounts with either "the" or "artist" in the username.
This works great!~ only problem is that the folder structure becomes popular all-time keyword username_[username] rather than just [username] like it would be with a normal rip command.... anyway to alter that? (atm I'm moving the files over afterwards manually but that prevent rerips from noticing a file already exists and skipping that download)
This works great!~ only problem is that the folder structure becomes popular all-time keyword username_[username] rather than just [username] like it would be with a normal rip command.... anyway to alter that? (atm I'm moving the files over afterwards manually but that prevent rerips from noticing a file already exists and skipping that download)
The first option for this would be to use symbolic links so that the "popular" folders all point to just [username], which is what I have been doing. Alternatively, perhaps the option extractor.*.directory might help (I haven't tried this myself though): https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst