unedit-for-reddit icon indicating copy to clipboard operation
unedit-for-reddit copied to clipboard

The Pushshift Data API is only accessible to moderators

Open DenverCoder1 opened this issue 2 years ago • 3 comments

The Pushshift Data API is only accessible to approved moderators as announced here.

Since Unedit and Undelete for Reddit relies on Pushshift to work, non-moderators can not use this extension unless an alternative data provider is found.

DenverCoder1 avatar May 29 '23 16:05 DenverCoder1

I know you've probably already seen this but RedArc was recently created as an API interface to access historical Pushshift dumps, I could look into how difficult it would be to change the API requests in the extension if you'd like?

RebootedDuck avatar May 31 '23 14:05 RebootedDuck

I know you've probably already seen this but RedArc was recently created as an API interface to access historical Pushshift dumps, I could look into how difficult it would be to change the API requests in the extension if you'd like?

Thanks, I heard about that, but as far as I know they aren't hosting an indexed API of all historical data.

All of the data is about 30 terabytes uncompressed which is not so simple to host or make accessible. I'd be happy to use an alternate API if someone has created one. I was able to find http://redarc.basedbin.org/ which has just an archive of just r/DataHoarder accessible via web requests, but not all subreddits.

DenverCoder1 avatar May 31 '23 19:05 DenverCoder1

Oh ok sorry I misunderstood then, judging from the recent announcement on r/pushshift and the MoU that they won't let us see it doesn't look good for pushshift ever coming back, also the site imploding on itself because of 3rd party apps

RebootedDuck avatar Jun 04 '23 11:06 RebootedDuck