OkCupid-Message-Downloader icon indicating copy to clipboard operation
OkCupid-Message-Downloader copied to clipboard

Update scraper for new message features and HTML format

Open lehrblogger opened this issue 6 years ago • 1 comments

As @5u2ie noted in https://github.com/lehrblogger/OkCupid-Message-Downloader/issues/43#issuecomment-414201276, the scraper no longer seems to be working. the error message is:

ERROR: There was an error queuing the threads to download - are you sure your username and password are correct? INFO: Done.

Thats logged by a somewhat-blind try except here, so it's hard to say exactly what the problem is. OkCupid has shipped several iterations of its messaging feature since this scraper was first created, so some of the assumptions about message types and formats might not be valid.

lehrblogger avatar Aug 20 '18 17:08 lehrblogger

I got a bit bored tonight so I figured I would look at this for a bit. It seems that the code is failing on line 137 of arrow_fetecher.py due to the following error:

"'NoneType' object has no attribute 'find_all'"

Seems that soup.find('ul', {'id': 'messages'}) no longer does what is intended. I am going to mess around a bit longer but can't make any promises that I will find the solution tonight.

JBarry465 avatar Sep 26 '18 00:09 JBarry465