search-engine-parser
search-engine-parser copied to clipboard
bisohns
Add Amazon
Hello! I am interested in adding this enhancement. Is the requirement something related to extracting the top ten results for a particular search? Can you kindly tell me regarding the requirement?
Yeah. Given a search on Amazon, it should return the titles, description, links, price and ratings for the search. Check out the Contribution guide for more details on contribution
I tried adding amazon. They have a stringent policy against web scraping. Every request returns an HTML page given below:
To discuss automated access to Amazon data, please contact [email protected]. For information about migrating to our APIs, refer to our Marketplace APIs at https://developer.amazonservices.com/ref=rm_c_sv, or our Product Advertising API at https://affiliate-program.amazon.com/gp/advertising/api/detail/main.html/ref=rm_c_ac for advertising use cases.
Enter the characters you see below Sorry, we just need to make sure you're not a robot. For best results, please make sure your browser is accepting cookies
Wow, that's some serious stuff. Maybe look into the headers that can be passed @devajithvs
Tried that too. Exactly copying the headers in the browser didn't work. I guess they have some other mechanism to prevent scraping.
Alright, I think we'll have to look into options of using selenium libraries tho. Meanwhile, I'll try out some additional headers and see where it gets us