Scraper only scrapes the first review page

[jimmy10023]

Hi, first of all thank you for the code!

I am however having the problem that when scraping multiple pages of reviews for the same product, only the first page gets scraped. The other pages get "scraped" too and show up in the data, but the actual reviews extracted from them are the ones from the first page.

Does anyone know how to fix this?

Thank you!

[hellochang]

Hi! I have the same issue, have you figured out how to fix it yet?

[karthikmagesan]

Hi, I am also facing same issue. Please let me know if you figured out how to fix it.

[cyanobrian]

I believe when you place the URLs in a TXT file, it reads the new line (\n) character when could mess with the URL. I found that if I strip off the last character of the URL being read or place the URLs in a python list, it worked fine for me.

[jmccaffrey]

I wanted to start with just an asin, get to the first page of reviews, and then keep going to the next page. You basically have to pull the url from next_page and loop on that

[ms-shashank]

Hi, first of all thank you for the code!

I am however having the problem that when scraping multiple pages of reviews for the same product, only the first page gets scraped. The other pages get "scraped" too and show up in the data, but the actual reviews extracted from them are the ones from the first page.

Does anyone know how to fix this?

Thank you!

The thing is Amazon restricts from scraping ,so when you make too many frequent requests this happens and scrapes only the first page and repeats this only so i would suggest use the request.sessions this might work and it worked for me