Dave Ascienzo

icon indicating an email [email protected]

icon company Applied Data Scientist @Thread-AI icon location New York, NY icon location Physics PhD turned data scientist, interested in machine learning, complex networks, reinforcement learning and other areas of applied mathematics.

Results 5 comments of Dave Ascienzo

ERROR: Spider must return Request, BaseItem, dict or None, got 'list'

Hello @maltokyo, I believe your issue should be fixed with the most recent pull request. The FormRequest will no longer be yielded as a list instance and you should be...

ERROR: Spider must return Request, BaseItem, dict or None, got 'list'

@maltokyo Is there any chance you can link to an example forum with similar results? It's hard to tell what the issue is from the logs alone, but looks like...

silently aborts before scraping all posts

@netllama Are you running this on a local machine or from a remote server? I ask because I ran into this issue running a crawl on an AWS EC2 instance,...

PostTime is always NULL

@netllama Hey, thanks for raising this issue! Are you still receiving this error? If so, can you please link an example page where this error is occurring? Thank you!

fails if no sub-forums

Hey @sliptonic , I can definitely work on fixing the scraper for cases where no child links are captured when starting from sub-forums. Downloading and storing images linked in threads...