gallery-dl icon indicating copy to clipboard operation
gallery-dl copied to clipboard

My DeviantArt description downloader won't download descriptions of text only deviations, is there anyway to fix this?

Open left1000 opened this issue 7 months ago • 9 comments

     "postprocessor":
    {
    
        "Itypethisviaacommandlineflagtoturniton":
        {
            "name": "metadata",
            "mtime": true,
            "mode": "custom",
            "event": "file,skip",
            "extension": "txt",
            "content-format"  : "{description}\n"

So, this will download descriptions for images, but it won't download descriptions when downloading .htm copies of text only deviations... is there a way to get it to continue working as it has, but also grab those text only deviation's descriptions?

left1000 avatar Jan 22 '24 06:01 left1000

To clarify the above processor creates an empty text file for the description download of any purely text based deviation post. For image deviation posts though this text file isn't empty and contains the proper description.

left1000 avatar Jan 22 '24 23:01 left1000

It doesn't seem like journal posts (I assume that's what you mean by "text-only") have a description, i.e. {description} is an empty string for them. I created a test journal post and there was no way to enter anything besides the main text field.

What would you expect description for these posts to be? Do you have an example where there is a description that somehow doesn't get returned as such by gallery-dl?

mikf avatar Jan 26 '24 18:01 mikf

Every deviation in this link https://www.deviantart.com/search?q=literature all written stories. The deviation is a written story. Then there's usually a small description below it.

Everything ABOVE the tags is the deviation post and is downloaded, everything below the tags is the description and does not get saved anywhere.

left1000 avatar Jan 27 '24 23:01 left1000

At least for this one or the stories here it extracts the full description text when metadata is enabled, so I'd guess it works in general.

$ gallery-dl -o metadata=true --filter 'print(description)' https://www.deviantart.com/desdemonadeblake/art/9-Steps-for-Adding-Genuine-Depth-to-Your-Story-518175360
<p align="LEFT">When writing a story, one of the most important ...

mikf avatar Jan 27 '24 23:01 mikf

Maybe I did not have metadata enabled?

followup... is -o metadata=true the same as -o metadata=1 ?

I've been using =1

left1000 avatar Jan 30 '24 00:01 left1000

No, it's not the same, in a technical sense. Some langs can convert it, but I would avoid it.

Hrxn avatar Jan 30 '24 08:01 Hrxn

Okay so that was not my issue,

.\gallery-dl -P NAMEOFPROCESSOR -o metadata=true -o skip="abort:20"

Is failing to get literature descriptions, it's just filling the literature description text file with blank nothing.

There must be something else wrong in my syntax? So,

       {             
           "filename": "{filename}.{extension}",
	    "cookies": "D:/gallery-dl/deviantart.com cookies.txt",
           "username": "<censored>",
           "password": "<censored>",	
           "refresh-token": "cache",
           "extra": true,
           "flat": true,
           "folders": false,
           "journals": "html",
           "mature": true,
           "metadata": false,
           "original": true,
           "quality": 100,
           "client-id": "<censored>",
           "client-secret": "<censored>",
           "wait-min": 0,
	    "postprocessors": [{
           "name": "mtime"
           }]
       },```
       
       
  ` "postprocessor":
   {
       "NAMEOFPROCESSOR":
       {
           "name": "metadata",
           "mtime": true,
           "mode": "custom",
           "event": "file,skip",
           "extension": "txt",
           "content-format"  : "{description}\n"
       },
       "another processornotinusehere": 
	{
           ""
       }
   }
}`

That is hopefully all relevant sections of .conf and such... anyways... can anyone find my error?

left1000 avatar Jan 31 '24 10:01 left1000

Did you check it with gallery-dl -K <your URL>? Maybe it's not {description}?

Hrxn avatar Jan 31 '24 11:01 Hrxn

The literature description does not appear at all on -K so I guess that's why it's not being downloaded, it doesn't exist at all????

edit: I guess this means that deviantart itself refers to descriptions on literature posts as something other than descriptions? and it's a field type gallery-dl doesn't comprehend or hasn't considered, so only a new feature that handles this type of information would help? maybe? I guess gallery-dl was never written with the deviantart literature section in mind?

left1000 avatar Feb 01 '24 20:02 left1000