gallery-dl icon indicating copy to clipboard operation
gallery-dl copied to clipboard

[Deviantart] Extended Metadata

Open vandervast opened this issue 5 months ago • 10 comments

Looking for a way to extract number of views of each deviation.

There's no mention of "views" or "view count" for DeviantArt in the docs, so I can only assume it's not supported. This would be a very useful metric to have access to, so I hope it'll included in the "stats" group along with favorites and comment count!

vandervast avatar Feb 08 '24 03:02 vandervast

Doesn't seem like DA exposes that info in either the -K output or the JSON, even the website doesn't offer it in any precision once 1k is reached, so either you'll have to scrape it off the website or forget about it.

rautamiekka avatar Feb 08 '24 12:02 rautamiekka

DA's private API seems to expose the exact view count as deviation -> stats -> views (https://www.deviantart.com/_puppy/dadeviation/init?deviationid=...), but gallery-dl currently uses DA's public API and that, as rautamiekka said, does not provide this information.

mikf avatar Feb 14 '24 13:02 mikf

Thank you for looking into this and explaining the nature of the problem! It's unfortunate that there's no easy solution, but at least the answers are here for anyone else wondering about this.

vandervast avatar Feb 15 '24 03:02 vandervast

DA's private API seems to expose the exact view count as deviation -> stats -> views (https://www.deviantart.com/_puppy/dadeviation/init?deviationid=...), but gallery-dl currently uses DA's public API and that, as rautamiekka said, does not provide this information.

This endpoint seems to provide exact views https://www.deviantart.com/developers/console/deviation/deviation_metadata/7824fc14d6fba6acbacca1cf38c24158

mdashlw avatar Feb 17 '24 03:02 mdashlw

But only when explicitly requesting them with ext_stats=1, which does not happen at the moment (current parameters). Time to extend DA's metadata option ...

mikf avatar Feb 17 '24 14:02 mikf

Is there any other metadata that's currently left out that you can include now?

Twi-Hard avatar Feb 17 '24 14:02 Twi-Hard

Quite a lot, it seems. All the ext_... parameters for /deviation/metadata are currently not enabled.

mikf avatar Feb 17 '24 14:02 mikf

Adding all of that extra metadata sounds really nice. :)

Edit: is it possible to make gallery-dl exit if the refresh token expires?

Twi-Hard avatar Feb 17 '24 16:02 Twi-Hard

This time I'm not going to wait several months before starting to work on this, don't worry.

Exiting when a refresh token is expired should be possible with the undocumented actions option.

    "actions": {
        "error:The refresh_token is invalid": "exit"
    }

mikf avatar Feb 17 '24 20:02 mikf

https://github.com/mikf/gallery-dl/commit/741fd00cec173bf7420c43349ec9050cf3227b06.

Should be fully functional, but I'm currently limiting the number of requested deviation objects to 10 when extended metadata is requested to make things easier. (DA allows fetching extended metadata of only 10 deviations at a time)

Regarding this issue's initial request: "metadata": "stats"

mikf avatar Feb 18 '24 22:02 mikf