newspaper4k
newspaper4k copied to clipboard
Set timeout for download_articles
Issue by yuhonglin
Thu Oct 6 01:09:43 2016
Originally opened as https://github.com/codelucas/newspaper/issues/293
Hi I am wondering what is the safest method to set a time limit for function download_articles, for example, in the following code,
paper = newspaper.build('cnn.com')
paper.download_articles(timeout=10) # for illustration
or like "http://stackoverflow.com/questions/2281850/timeout-function-if-it-takes-too-long-to-finish", using
with timeout(seconds=100):
paper.download_articles()
And if I set the timeout following method this way, will the undownloaded url be downloaded next time (after rebuilding the paper)?
Thanks!