gallery-dl icon indicating copy to clipboard operation
gallery-dl copied to clipboard

[deviantart] can't download galleries anymore, worked fine when i first got it

Open Shadow-night45 opened this issue 9 months ago • 16 comments

in the last couple of days, i could not download galleries from deviantart anymore, when I first got gallery-dl over a week ago, I was able to download galleries from deviantart just fine, only recently I could not do that anymore and kept getting this

[deviantart][warning] API responded with 429 Too Many Requests. Using (number)s delay.

i even waited like 30 minutes on a gallery for a single image to get downloaded but nothing, just kept showing that,

other sites that gallery-dl support it downloads just fine but not for deviantart recently, I've been trying to figure out how to fix this but can't find a solution, either there is a fix to this and I'm just blind to see it because I don't understand this program/coding stuff, or is something on deviantart end that is causing it or a recent update with gallery-dl caused something

I would like some help with this please

Shadow-night45 avatar May 06 '24 16:05 Shadow-night45

Are you using your own OAuth credentials? I can't tell, you did not provide a verbose log.

Hrxn avatar May 06 '24 16:05 Hrxn

Are you using your own OAuth credentials? I can't tell, you did not provide a verbose log.

i don't know what that is nor do I know how to work with that and what is a verbose log?

Shadow-night45 avatar May 06 '24 19:05 Shadow-night45

Are you using your own OAuth credentials? I can't tell, you did not provide a verbose log.

do you mean this for a verbose log? (the gallery that I'm downloading is an example of the problem I'm having)

https://imgur.com/yAHTSqB

Shadow-night45 avatar May 06 '24 19:05 Shadow-night45

Almost. Verbose log means you also include the --verbose flag (or the -v short form).

But it's already obvious that you are using the built-in default credentials, which are well beyond limit already, it seems.

You need your own tokens to access the API again reliably: https://gdl-org.github.io/docs/configuration.html#api-tokens-ids

Hrxn avatar May 06 '24 20:05 Hrxn

Almost. Verbose log means you also include the --verbose flag (or the -v short form).

But it's already obvious that you are using the built-in default credentials, which are well beyond limit already, it seems.

You need your own tokens to access the API again reliably: https://gdl-org.github.io/docs/configuration.html#api-tokens-ids

ok I think I did all the steps in the link your provided for deviantart but I'm still getting the errors, still cant download stuff

and I'm not understanding this "verbose log"

Shadow-night45 avatar May 06 '24 20:05 Shadow-night45

The output of gallery-dl --verbose <YOUR URL HERE>

Also, please don't post screenshots or something, just paste the text output here. Fenced between triple backticks:

Three ` characters
<YOUR LOG>
Three ` characters

Hrxn avatar May 06 '24 20:05 Hrxn

im still not understanding, because i dont understand coding stuff, all i want to do is try and fix this downloading issue with deviantart

Shadow-night45 avatar May 06 '24 21:05 Shadow-night45

Then you probably did the steps you said you did wrong..

Hrxn avatar May 06 '24 21:05 Hrxn

i dont know what i did wrong, I'm trying to get help with this because I cant figure this out on my own

all that It keeps doing is asking about (Register your own OAuth application and use its credentials to prevent this error:) then just spams (API responded with 429 Too Many Requests)

Shadow-night45 avatar May 06 '24 21:05 Shadow-night45

Paste the relevant part of your config here (redact the tokens). Or the entire config.

Hrxn avatar May 06 '24 21:05 Hrxn

in the last couple of days, i could not download galleries from deviantart anymore, when I first got gallery-dl over a week ago, I was able to download galleries from deviantart just fine, only recently I could not do that anymore and kept getting this

[deviantart][warning] API responded with 429 Too Many Requests. Using (number)s delay.

i even waited like 30 minutes on a gallery for a single image to get downloaded but nothing, just kept showing that,

Try the solution from here https://github.com/mikf/gallery-dl/issues/5558

unmnae avatar May 06 '24 21:05 unmnae

Paste the relevant part of your config here (redact the tokens). Or the entire config.

how do I do that?

Shadow-night45 avatar May 06 '24 22:05 Shadow-night45

in the last couple of days, i could not download galleries from deviantart anymore, when I first got gallery-dl over a week ago, I was able to download galleries from deviantart just fine, only recently I could not do that anymore and kept getting this [deviantart][warning] API responded with 429 Too Many Requests. Using (number)s delay. i even waited like 30 minutes on a gallery for a single image to get downloaded but nothing, just kept showing that,

Try the solution from here #5558

I followed everything on the configuration from this https://gdl-org.github.io/docs/configuration.html#api-tokens-ids

the only part I'm confused on is (get a new refresh-token for the new client-id (gallery-dl oauth:deviantart)

when I do that is brings me to a new tab that says

[Your 'refresh-token' is

(the numbers and letters that I wont say)

This value has been cached and will automatically be used.]

after that I still try to do the (gallery-dl [the link of the person gallery] )

which keeps giving me what I said at the beginning

Shadow-night45 avatar May 06 '24 22:05 Shadow-night45

Okay, here's an entire config file for DeviantArt, you can copy this and use it as a starting point for your own customizations:

{
    "extractor":
    {
        "deviantart":
        {
            "gallery":
            {
                "archive-format": "g_{username}_{index}.{extension}",
                "directory": ["DeviantArt", "+Galleries", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{author[username]}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },
            "folder":
            {
                "archive-format": "F_{folder[uuid]}_{index}.{extension}",
                "directory": ["DeviantArt", "+Galleries", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{folder[owner]}", "Folders", "{folder[title]}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },
            "favorite":
            {
                "archive-format": "f_{username}_{index}.{extension}",
                "directory": ["DeviantArt", "+Favorites", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{username}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },
            "collection":
            {
                "archive-format": "C_{collection[uuid]}_{index}.{extension}",
                "directory": ["DeviantArt", "+Favorites", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{collection[owner]}", "Collections", "{collection[title]}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },

            "archive-prefix": "",
            "archive": "D:\\gallery-dl\\archive\\gallery-dl.archive.deviantart.db",

            "directory": ["DeviantArt", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{username}"],
            "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}",

            "jwt": false,
            "client-id": "<YOUR_CLIENT_ID>",
            "client-secret": "<YOUR_CLIENT_SECRET>",
            "refresh-token": "<YOUR_REFRESH_TOKEN>",

            "auto-watch": false,
            "auto-unwatch": false,
            "comments": false,
            "extra": false,
            "folders": false,
            "group": true,
            "include": "gallery",
            "journals": "html",
            "metadata": true,
            "pagination": "manual",
            "public": false,

            "flat": true,
            "mature": true,
            "original": true,
            "quality": "png",

            "skip": true,
            "wait-min": 0,
            "sleep-request": [0.8, 1.2],
            "postprocessors": ["nonpicsortnontagged"]
        }
    }
}

You need to replace "client-id", "client-secret", and "refresh-token" with your own values and you're ready to go.

Hrxn avatar May 06 '24 22:05 Hrxn

Okay, here's an entire config file for DeviantArt, you can copy this and use it as a starting point for your own customizations:

{
    "extractor":
    {
        "deviantart":
        {
            "gallery":
            {
                "archive-format": "g_{username}_{index}.{extension}",
                "directory": ["DeviantArt", "+Galleries", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{author[username]}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },
            "folder":
            {
                "archive-format": "F_{folder[uuid]}_{index}.{extension}",
                "directory": ["DeviantArt", "+Galleries", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{folder[owner]}", "Folders", "{folder[title]}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },
            "favorite":
            {
                "archive-format": "f_{username}_{index}.{extension}",
                "directory": ["DeviantArt", "+Favorites", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{username}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },
            "collection":
            {
                "archive-format": "C_{collection[uuid]}_{index}.{extension}",
                "directory": ["DeviantArt", "+Favorites", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{collection[owner]}", "Collections", "{collection[title]}"],
                "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}"
            },

            "archive-prefix": "",
            "archive": "D:\\gallery-dl\\archive\\gallery-dl.archive.deviantart.db",

            "directory": ["DeviantArt", "{bkey|'Unsorted'}", "{ckey|''}", "{tkey|''}", "{skey|''}", "{mkey|''}", "{username}"],
            "filename": "{date:%Y-%m-%d}.{index}.{title!t:?/./R.//}{author[username]}.{extension}",

            "jwt": false,
            "client-id": "<YOUR_CLIENT_ID>",
            "client-secret": "<YOUR_CLIENT_SECRET>",
            "refresh-token": "<YOUR_REFRESH_TOKEN>",

            "auto-watch": false,
            "auto-unwatch": false,
            "comments": false,
            "extra": false,
            "folders": false,
            "group": true,
            "include": "gallery",
            "journals": "html",
            "metadata": true,
            "pagination": "manual",
            "public": false,

            "flat": true,
            "mature": true,
            "original": true,
            "quality": "png",

            "skip": true,
            "wait-min": 0,
            "sleep-request": [0.8, 1.2],
            "postprocessors": ["nonpicsortnontagged"]
        }
    }
}

You need to replace "client-id", "client-secret", and "refresh-token" with your own values and you're ready to go.

where would I put this because I'm using notepad for the editing of this? https://github.com/mikf/gallery-dl/blob/master/docs/gallery-dl.conf but I'm not sure where I'm supposed to put it after downloading it?

and is the client-id 5-digit numbers and the client-secret 30+ digit with numbers and letters?

and is the refresh-token the one that I get from the new tab?, the one that says

(Your 'refresh-token' is

(REDACTED)

This value has been cached and will automatically be used.)

because I don't want to screw something up and end up going back to square one

Shadow-night45 avatar May 06 '24 22:05 Shadow-night45

Here are the config locations:

https://github.com/mikf/gallery-dl#locations

Just create one (!) file in whichever location you prefer for your config file, you can do this with notepad, or notepad++ or something.

And yes, that token from the new tab is the "refresh-token" you'll need.

Hrxn avatar May 07 '24 02:05 Hrxn