redlib-org
Handle URL/subreddit subscription limit
Is there a max number of subreddits one can use in redlib?
Currently on latest version (just re-pulled, same issue)
Along with the failed to parse message, I also receive a "URI Too Long" error once adding additional subreddits. If I remove the subreddits from my docker container and restart it, the error goes away and redlib works again.
Not sure how many subreddits tipped it over the edge, but I was attempting to use a total of 936 subreddits when I encountered the issue. Cutting the number in half to 468 resolved the URI length message.
Assuming this limit would be on reddit's side, not anything redlib could work around without sending multiple requests after x subreddits have been added.
redlib | ERROR redlib::client > Got an invalid response from reddit expected value at line 1 column 1. Status code: 414 URI Too Long redlib | ERROR redlib::utils > Error page rendered: Failed to parse page JSON data: expected value at line 1 column 1
I've also attached my .env file with the full 936 subreddits.
Originally posted by @scriptsandthings in #324
Thx for opening this and being so responsive.
I didn't notice last evening that in the redlib error screenshot there's some subreddit names that don't exist.
removing the + and separating them by line made it easier to search through and verify.
boatbuubackblaze (lmao supposed to be /r/boatbuilding and r/backblaze) hit me in the face, then treens (should be /r/trees)
Can duplicate the error by going to https://old.reddit.com/subreddits/mine and clicking the "multireddit of my subscriptions" link which produces the error directly in reddit. I'd just copied it direct into the text editor to get the part of the URL i needed (the subreddit list) w/o clicking it previously.
None of the subreddits appear wrong on the subreddits/mine page, reddit itself is generating that multireddit link and bugging out.
I'd mark this closed, not a redlib issue.
Issue is using https://old.reddit.com/subreddits/mine and that multireddit link, then pulling subscriptions from that is not a good way to export subs for use anywhere
I think this is still a relevant issue. The error code being 414 specifically makes me think it's still an error in the way we send subscription URLs - if there were any missing subs listed, they just get skipped by the multi-sub handling code, serverside. For example: https://rl.bloat.cat/r/rust+asfdashdjfg
Ok cool. I'll parse and correct my subreddit list. This bug (and my reluctance to spinning up multiple instances of redlib for different subreddits) is all that's keeping me locked with the Reddit app.
Thanks!!
and my reluctance to spinning up multiple instances of redlib for different subreddits
I haven't tested this, but you might be able to use the same instance for that by using a reverse proxy to expose the instance under two different ports. That way, your browser will keep separate cookie stores for each address.
and my reluctance to spinning up multiple instances of redlib for different subreddits
I haven't tested this, but you might be able to use the same instance for that by using a reverse proxy to expose the instance under two different ports. That way, your browser will keep separate cookie stores for each address.
I'd still need two docker containers and two urls setup tho? The end goal is to just have one big ass list of subs in a single container. Container logs shows issues too after whatever subreddit list number I crossed
I've also run into this bug now, it's triggered based on the number of subscriptions. Since Reddit doesn't have a subscription limit, it seems likely this bug is caused by the way redlib is interacting with Reddit.
A tangential issue here is that the error message is quite unhelpful, and misleadingly points towards the Reddit rate-limit / IP blocking issue. The only way to see the real issue is via container logs.