[bluesky] add '-o expand=1' support
if a post has replyCount>0 but no replies, that means the bluesky appview is restricting the depth of the thread.
not everyone wants to recurse over all posts, so leave it as -o expand=1 like the twitter extractor.
testing like this under an extremely long root thread: env/bin/gallery-dl --verbose -o expand=1 -J 'https://bsky.app/profile/blackle.bsky.social/post/3lpmnsdic722n'
Any particular reason why you are collecting all posts first, then expanding them, and only then returning any results instead of doing "lazy" expansions for each returned post as is the case for Twitter?
there wasn't any intentional technical reason other than not being super familiar with the codebase so I wanted to isolate all the logic away, but I do need access to bluesky's threadViewPost so I can fetch the "replies" field and I saw some other bluesky functions strip it (via post = post["post"]), so if there's a way to access replies with "lazy" expansions I'm interested
regarding lazy expansion, do you mean this part of logic on the twitter extractor?
def _expand_tweets(self, tweets):
seen = set()
for tweet in tweets:
obj = tweet["legacy"] if "legacy" in tweet else tweet
cid = obj.get("conversation_id_str")
if not cid:
tid = obj["id_str"]
self.log.warning(
"Unable to expand %s (no 'conversation_id')", tid)
continue
if cid in seen:
self.log.debug(
"Skipping expansion of %s (previously seen)", cid)
continue
seen.add(cid)
try:
yield from self.api.tweet_detail(cid)
except Exception:
yield tweet
specifically yield from ... and how I don't use that in bluesky expansion logic, or something else entirely?