redditDataExtractor icon indicating copy to clipboard operation
redditDataExtractor copied to clipboard

Published 20 hours ago • verified publisher icon NSchrading

Only allowed to download max 1000 posts?

Open pangjac opened this issue 9 years ago • 2 comments

In settings there is "Max Posts Retrieved in Subreddit Content Download[1-1000]", so if I want to download posts more than 1000? 1000 posts is far from enough to get data analysis.

Which setting will allow me to extract next 1001-2000, 2001-3000 etc posts? Is there an automatically mechansim to download ALL posts for one reddit topic?

Thanks!

pangjac avatar Oct 08 '15 23:10 pangjac

Sorry, currently the Reddit API restricts downloads to the most recent 1000 submissions. If you want to download past the 1000 limit, you have to write your own scraper that can pass in time values to Reddit. Currently this isn't something available in my tool, and I haven't seen anything that does this for you, but if you look around you might be able to find something.

NSchrading avatar Oct 12 '15 04:10 NSchrading

Dear J, thanks for the information. I think another way is just scrape the website directly, bypassed an reddit API?

pangjac avatar Oct 15 '15 17:10 pangjac