internal-displacement icon indicating copy to clipboard operation
internal-displacement copied to clipboard

Deal with datetime issue

Open georgerichardson opened this issue 7 years ago • 4 comments

Sometimes no publication date is available and a blank string is returned. However the db model expects a date time.

Possible fix in scraper.Scraper.html_article:

if not isinstance(a.publish_date, datetime.datetime):
    article_pub_date = datetime.datetime.now()
else:
    article_pub_date = a.publish_date

georgerichardson avatar May 04 '17 18:05 georgerichardson

Does it make more sense to use None/NULL? It seems misleading to say that the publication date of an article is when the scraper ran.

WanderingStar avatar May 04 '17 18:05 WanderingStar

I think None makes more sense. If there had to be a date, a fallback could be the latest date mentioned in the article ( or the latest Report).

domingohui avatar May 05 '17 00:05 domingohui

Yeah agreed None makes more sense now that I think about it.

simonb83 avatar May 17 '17 23:05 simonb83

Yep!

georgerichardson avatar May 18 '17 21:05 georgerichardson