json.decoder.JSONDecodeError

[mikan01]

Hi,

I keep getting this error after scraping few books (3 to 4 books). Please help

Traceback (most recent call last): File "main.py", line 62, in <module> audio_files = scraper.scrape_book_audio(driver, book_json) File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 223, in scrape_book_audio audio_url = audio_request.json()['url'] File "C:\Python38\lib\site-packages\requests\models.py", line 888, in json return complexjson.loads( File "C:\Python38\lib\json\__init__.py", line 357, in loads return _default_decoder.decode(s) File "C:\Python38\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Python38\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[leoncvlt]

I could not replicate that, but the latest commit 43aa26225c8c29474e21939ae5e5c1981461064a should at least make it fail gracefully and not stop the scraping process.

I'll leave this open for now - let me know if you figure a way to consistently replicate that or if it fails with a specific book in particular.

[mikan01]

It's still failing. :\

I am able to download few books 1 or 2 at a time then process gets stopped with this error, then I have rerun the command to scrape more books.

Traceback (most recent call last): File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 229, in scrape_book_audio audio_request_json = audio_request.json(); File "C:\Python38\lib\site-packages\requests\models.py", line 888, in json return complexjson.loads( File "C:\Python38\lib\json_init_.py", line 357, in loads return _default_decoder.decode(s) File "C:\Python38\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Python38\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "main.py", line 66, in audio_files = scraper.scrape_book_audio(driver, book_json, args.language) File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 239, in scrape_book_audio print(f'[!] Received malformed json data: {audio_request_json.text}') UnboundLocalError: local variable 'audio_request_json' referenced before assignment

[Wunderharke]

It's still failing. :\

I am able to download few books 1 or 2 at a time then process gets stopped with this error, then I have rerun the command to scrap more books.

Traceback (most recent call last): File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 229, in scrape_book_audio audio_request_json = audio_request.json(); File "C:\Python38\lib\site-packages\requests\models.py", line 888, in json return complexjson.loads( File "C:\Python38\lib\json__init__.py", line 357, in loads return _default_decoder.decode(s) File "C:\Python38\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Python38\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "main.py", line 66, in audio_files = scraper.scrape_book_audio(driver, book_json, args.language) File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 239, in scrape_book_audio print(f'[!] Received malformed json data: {audio_request_json.text}') UnboundLocalError: local variable 'audio_request_json' referenced before assignment

I am getting the same error when scraping the second book on a new clone on ca0d913396a34cf2792d231a86d7c7c11ad2501d.

[Wunderharke]

I might found the cause:

The error occurs after accepting the cookies inside the ChromeDriver window. It seems there is some interference, maybe with some json data on side-loaded sites, just guessing ...

When not interacting with the Chrome window, the program seems to work like expected.

[leoncvlt]

My bad, latest commit 426d5bdc854dba008fc1674357f9199be06c9d0d should fix the handling of the exception. Again, it will not fix the bug but at least it should handle it without stopping the program and provide some more information about what's failing.

@Wunderharke Interesting! I never touch the Chrome window while scraping anyway, since it's all automated - just let it do its job. @mikan01 can you confirm if that makes it better for you too?

[mikan01]

My bad, latest commit 426d5bd should fix the handling of the exception. Again, it will not fix the bug but at least it should handle it without stopping the program and provide some more information about what's failing.

@Wunderharke Interesting! I never touch the Chrome window while scraping anyway, since it's all automated - just let it do its job. @mikan01 can you confirm if that makes it better for you too?

Thank you, now the problem seems to be fixed. :)

[rocketinventor]

@mikan01, If this issue has been resolved would you please consider closing it?

Thanks!