bangalore-flat-and-flatmates-crawler
bangalore-flat-and-flatmates-crawler copied to clipboard
Repeating text content
There seems to be repeating text content for some entries in the database. The entire body of text just repeats itself after ending. Can be fixed by checking by:
length = len(content)
if content[:length//2] == content[length//2:]:
content = content[:length//2]
This needs to be added to the processor