amazon-review-scraper icon indicating copy to clipboard operation
amazon-review-scraper copied to clipboard

TypeError: 'NoneType' object is not iterable

Open wolleakira opened this issue 5 years ago • 4 comments

I installed all the requirements and ran the code for testing without changing the urls or anything and got the following error:

Downloading https://www.amazon.com/Nike-Womens-Reax-Running-Shoes/product-reviews/B07ZPL752N/ref=cm_cr_dp_d_show_all_btm?ie=UTF8&reviewerType=all_reviews Traceback (most recent call last): File "C:/Users/amazon-review-scraper-master/reviews.py", line 46, in for r in data['reviews']: TypeError: 'NoneType' object is not iterable

Could you tell me where the code might need to be changed or what I should do?

wolleakira avatar May 22 '20 11:05 wolleakira

I have the same issue! It used to work fine (a few weeks ago) but I see the same error right now. Would you great if anybody can help fixing this?

christiefung avatar Jun 08 '20 20:06 christiefung

Looks like Amazon has begun blocking these requests. The Robots.txt file shows the customer reviews are Disallowed. They appear to be pushing folks to the Product Advertising API which requires you to be a seller or have a e-commerce site to pull the data. Hopefully I'm wrong, but based on the information that came back on the query it appears that is what is going on.

<div class="a-row a-spacing-double-large" style="width: 350px; margin: 0 auto">

    <div class="a-row a-spacing-medium a-text-center"><i class="a-icon a-logo"></i></div>

    <div class="a-box a-alert a-alert-info a-spacing-base">
        <div class="a-box-inner">
            <i class="a-icon a-icon-alert"></i>
            <h4>Enter the characters you see below</h4>
            <p class="a-last">Sorry, we just need to make sure you're not a robot. For best results, please make sure your browser is accepting cookies.</p>
            </div>
        </div>

jabney27 avatar Jun 11 '20 17:06 jabney27

Definitely getting blocked here. You may want to change the User-Agent

scrapehero-code avatar Jun 21 '20 13:06 scrapehero-code

In reviews.py if we add one more condition in if then It will work fine because if verified purchase is blank(None) then it will show error. If we add a condition in if block to check that, the error will be solve. changes:

    if 'verified' in r and r['verified'] is not None:
        if 'Verified Purchase' in r['verified']:
            r['verified'] = 'Yes'
       else:
            r['verified'] = 'Yes'
   else:
       r['verified'] = 'No'

You can see I add one more condition in first If and also add a else for outer If.

rishab05 avatar Aug 13 '20 07:08 rishab05