UltimaScraper
UltimaScraper copied to clipboard
[Tutorial] Installation in macOS
So I had a few problems installing on macOS 12.7 because of psycopg2
so I've decided to do a little tutorial:
This is how you install the UltimaScraper on macOS:
Prep
- Install Python 3.10 from here: https://www.python.org/downloads/release/python-31011/ (Be advised, this is an old version as newer 3.10 releases do not have installers for macOS)
- Download Postgres from here: https://postgresapp.com/downloads.html and place it into
/Applications
- Download UltimaScraper and unzip it to
~/Documents/UltimaScraper-main
- Open up a terminal window and do the following:
-
cd ~
followed byvi .zshrc
hit "i" and add this line into the fileexport PATH="/Applications/Postgres.app/Contents/Versions/latest/bin:$PATH"
to save type:wq
- Use the command
source .zshrc
to reload your Terminal config. - Install Poetry
pipx install poetry
Installation:
- Go into the UltimaScraper Path
cd ~/Documents/UltimaScraper-main
- Use poetry to install everything
poetry install
- Follow the tutorial from @DIGITALCRIMINAL here https://github.com/DIGITALCRIMINAL/UltimaScraper#mandatory-tutorial
- Be advised the default auth.json will be not enabled and you need to enable it. To do this look at the last line where it says
"active": null
this should be"active": true
- Start UltimaScraper with
poetry run python start_us.py
Config:
Customize you config with these settings:
https://github.com/DIGITALCRIMINAL/UltimaScraper#optional
With that everything worked for me depending on your installations with pip you might not run into the same problem as me with a fresh macOS 12.7 install.
Thanks for the tutorial @Gixxa42 - I managed to get everything installed but it looks like I'm running on a diff version of Python, does that matter? The scraper will not respond after site selection... it just hangs at this. Have any idea how to resolve?
@igobytwo
Yes, it matters, as is stated on the front page you need Python 3.10.1+. Looks like you are using 3.12 or 3.11 both will not work.
@igobytwo
Yes, it matters, as is stated on the front page you need Python 3.10.1+. Looks like you are using 3.12 or 3.11 both will not work.
@Gixxa42 ah - seems that i get the same hang up. I've uninstalled other versions of the Python and still just sits after I chose the site...
@igobytwo
It still says that you are using python3.11 (3.11.4).
@igobytwo
It still says that you are using python3.11 (3.11.4).
@Gixxa42 oops sorry wrong screengrab. It's 3.10.11
@igobytwo
Hmm not sure what is causing that. Did you rerun pipx install poetry
and poetry install
with python 3.10.11?
@igobytwo
Hmm not sure what is causing that. Did you rerun
pipx install poetry
andpoetry install
with python 3.10.11?
@Gixxa42 I did. still hangs up in the same spot. I recently migrated to a new computer and it was working fine on my last... not sure what changed. sucks because i don't even get an error message it just hangs after i select a site. anyway, i appreciate the help.
@yamaxiaofu
You need to install pipx first --> pip3.10 install pipx
- cd ~Documents/UltimaScraper-main
@yamaxiaofu
my mistake, should've been cd ~/Documents/UltimaScraper-main
I've had a million and one issues with Poetry recently.
I'm also having the same issue where it just hangs...
@RK625 This looks more like a bug with your environment and is beyond the scope of this tutorial. At the moment I'm using Python 3.10.13 via homebrew and it still works for me.
The only problem I'm currently having is I get return self.directory_manager_users[user_id]
in every scrape. I would say this is because of DRM and PPVs that I bought.
Can you please help me? I tried to understand what you are writing but it was really hard for me. Can you make a screen recording how to install please ? Really need a video step by step.
Hi @msrken
I no longer use this scraper but of-scraper (https://github.com/datawhores/OF-Scraper) I suggest you do the same as the install is easier and it has a better documentation.