internal-displacement icon indicating copy to clipboard operation
internal-displacement copied to clipboard

Scraping reliability score

Open georgerichardson opened this issue 8 years ago • 1 comments

Write a function in article.Article that calculates the percentage of scraped fields which are returned empty.

We may consider expanding the definition of scraping reliability later, so suggestions welcome.

georgerichardson avatar Feb 18 '17 16:02 georgerichardson

We can exploit '''vars()''' for this end:

## loop through each self variable in article class and test they're not 0, False, or None

def reliability(article):
	return (len([v for v in vars(article).values() if(v)])/
		    len(vars(article)))

ghost avatar Mar 02 '17 06:03 ghost