internal-displacement
internal-displacement copied to clipboard
Deal with datetime issue
Sometimes no publication date is available and a blank string is returned. However the db model expects a date time.
Possible fix in scraper.Scraper.html_article
:
if not isinstance(a.publish_date, datetime.datetime):
article_pub_date = datetime.datetime.now()
else:
article_pub_date = a.publish_date
Does it make more sense to use None/NULL? It seems misleading to say that the publication date of an article is when the scraper ran.
I think None makes more sense. If there had to be a date, a fallback could be the latest date mentioned in the article ( or the latest Report).
Yeah agreed None makes more sense now that I think about it.
Yep!