subreddit-comments-dl icon indicating copy to clipboard operation
subreddit-comments-dl copied to clipboard

can't scrape past a certain date

Open gtrane opened this issue 1 year ago • 5 comments

Hi, First of all, thank you for this great program! I have used your code successful for scraping a subreddit from specific utc date ranges. However, I have encountered a problem where I can't scrape anything past the UTC: 1670743183

my input to terminal: python src/subreddit_downloader.py --reddit-id --reddit-secret --reddit-username --debug --batch-size 500 --utc-after 1670743183

The error is below. I have no idea why this is occurring, any advice would be greatly appreciated! Thank you.

subreddit_downloader.py 308 typer.run(main)

main.py 859 run app()

main.py 214 call return get_command(self)(*args, **kwargs)

core.py 829 call return self.main(*args, **kwargs)

core.py 782 main rv = self.invoke(ctx)

core.py 1066 invoke return ctx.invoke(self.callback, **ctx.params)

core.py 610 invoke return callback(*args, **kwargs)

main.py 497 wrapper return callback(**use_params) # type: ignore

contextlib.py 79 inner return func(*args, **kwds)

subreddit_downloader.py 299 main assert utc_lower_bound < utc_upper_bound, f"utc_lower_bound '{utc_lower_bound}' should be " \

TypeError: '<' not supported between instances of 'NoneType' and 'str'

gtrane avatar Feb 24 '23 20:02 gtrane