scrapeadvisor
scrapeadvisor copied to clipboard
A user-friendly python-based GUI which provides sentiment analysis of users' reviews toward a specific TripAdvisor facility
scrapeadvisor
A user-friendly python-based GUI which provides sentiment analysis of users' reviews toward a specific TripAdvisor facility
Table of Contents
- About
- Getting Started
- Run
- Usage
- Statistics
- Supported Languages
- Built Using
- Authors
- Acknowledgments
- Disclaimer
About
If you're reading, dear Tripadvisor, Inc., hire me!
Getting Started
Prerequisites
Installing
Make sure you've all Python dependencies installed with:
scrape@advisor:~$ pip3 install -r requirements.txt
Also, the following R packages are needed:
- dplyr
- readr
- lubridate
- ggplot2
- tidytext
- tidyverse
- stringr
- tidyr
- scales
- broom
- purrr
- widyr
- igraph
- ggraph
- SnowballC
- wordcloud
- reshape2
- TeachingDemos
You can manually install missing ones with:
install.packages("library_name")
or run this script (credit to @stevenworthington) to install them all.
Note
For Italian language support, TextWiller library must be installed.
To do that:
install.packages("devtools")
install_github("livioivil/TextWiller")
Run
In order to launch scrapeadvisor GUI, run:
scrape@advisor:~$ python3 ScrapeAdvisor.py
Usage
Insert URL
- Insert the main page URL of a TripAdvisor structure (pub/restaurant/hotel/whatever) in the URL bar and click Enter (or press Enter)
- Wait until "Reviews Loaded" label appears (may take time, depending on number of reviews)
Show Reviews
After the download is finished, press "Show Reviews" to see all the downloaded reviews.
Sentiment Analysis
After the download is finished, press "Sentiment Analysis" button and wait: all the graphs related to the facility will appear follow after, so you can swipe between them.
Statistics
- Frequent couple/trio of consecutive words (bigrams/trigrams)
- Most positive/negative review
- Top positive/negative sentiments of users
- The trending/shrinking words
- Users' main cities
Screenshot
Supported Languages
- English
- Italian
Built Using
- Kivy - GUI
- Beautiful Soup - HTML scraping
- R - Sentiment Analysis
Acknowledgements
- @susanli2016 - Web Scraping TripAdvisor
- TextWiller - For providing Italian stop words and lexicon
- All the other packages - Thank you for being you
Disclaimer
Scrapeadvisor is provided under this License on an AS-IS basis, without warranty of any kind, either expressed, implied, or statutory, including, without limitation, warranties that the scrapeadvisor is free of defects, merchantable, fit for a particular purpose or non-infringing.
To the extent permitted under Law, scrapeadvisor is provided under an AS-IS basis. The scrapeadvisor Team shall never, and without any limit, be liable for any damage, cost, expense or any other payment incurred as a result of scrapeadvisor's actions, failure, bugs and/or any other interaction between scrapeadvisor and end-equipment, computers, other software or any 3rd party, end-equipment, computer or services.
We do not encourage running scrapeadvisor against Tripadvisor without prior mutual consent. The scrapeadvisor Team accept no liability and are not responsible for any misuse or damage caused by scrapeadvisor.