fastquant icon indicating copy to clipboard operation
fastquant copied to clipboard

Published 20 hours ago • verified publisher icon enzoampil

News article scraper

Open jpdeleon opened this issue 5 years ago • 3 comments

Newspaper seems like a good tool to scrape and curate articles related to PSE-listed stocks.

I can imagine using a different tool to search for recent news related to a company and using newspaper to scrape that article.

What do you think?

Edit:

  • See also: https://github.com/topics/news-scraper

jpdeleon avatar Jan 14 '20 13:01 jpdeleon

@jpdeleon I think this makes sense to do - was intending to do something similar with Twitter feeds. However, do take note it will only add value as a data source if we create a pipeline that pulls new PSE related article URLs, perform scraping on those URLs, and then store the data on a separate DB. I do agree that integrating the functions to do this into psequant would make sense.

Glad to have further discussions on how to implement this, specifically on how to get PSE related article URLs so that we can use newspaper to do the actual scraping :)

enzoampil avatar Jan 14 '20 15:01 enzoampil

@rafmacalaba I added a link above about existing news scrapers. Please let me know which is best for our purpose.

jpdeleon avatar Jun 14 '20 04:06 jpdeleon

@jpdeleon newspaper looks promising!

rafmacalaba avatar Jun 14 '20 11:06 rafmacalaba