gallery-dl icon indicating copy to clipboard operation
gallery-dl copied to clipboard

[bluesky] add '-o expand=1' support

Open lun-4 opened this issue 7 months ago • 2 comments

if a post has replyCount>0 but no replies, that means the bluesky appview is restricting the depth of the thread.

not everyone wants to recurse over all posts, so leave it as -o expand=1 like the twitter extractor.

testing like this under an extremely long root thread: env/bin/gallery-dl --verbose -o expand=1 -J 'https://bsky.app/profile/blackle.bsky.social/post/3lpmnsdic722n'

lun-4 avatar May 28 '25 17:05 lun-4

Any particular reason why you are collecting all posts first, then expanding them, and only then returning any results instead of doing "lazy" expansions for each returned post as is the case for Twitter?

mikf avatar Jun 02 '25 17:06 mikf

there wasn't any intentional technical reason other than not being super familiar with the codebase so I wanted to isolate all the logic away, but I do need access to bluesky's threadViewPost so I can fetch the "replies" field and I saw some other bluesky functions strip it (via post = post["post"]), so if there's a way to access replies with "lazy" expansions I'm interested

lun-4 avatar Jun 02 '25 20:06 lun-4

regarding lazy expansion, do you mean this part of logic on the twitter extractor?

    def _expand_tweets(self, tweets):
        seen = set()
        for tweet in tweets:
            obj = tweet["legacy"] if "legacy" in tweet else tweet
            cid = obj.get("conversation_id_str")
            if not cid:
                tid = obj["id_str"]
                self.log.warning(
                    "Unable to expand %s (no 'conversation_id')", tid)
                continue
            if cid in seen:
                self.log.debug(
                    "Skipping expansion of %s (previously seen)", cid)
                continue
            seen.add(cid)
            try:
                yield from self.api.tweet_detail(cid)
            except Exception:
                yield tweet

specifically yield from ... and how I don't use that in bluesky expansion logic, or something else entirely?

lun-4 avatar Aug 05 '25 02:08 lun-4