shot-scraper

shot-scraper copied to clipboard

---- 📚 Documentation preview 📚: https://shot-scraper--139.org.readthedocs.build/en/139/

iLLumi-420

add more ways to configure web browser from command-line

4

* option to use system browser * option to add custom browser command-line arguments * option to ignore https headers ---- :books: Documentation preview :books:: https://shot-scraper--91.org.readthedocs.build/en/91/

looran

Failing test: Selector with a wait

1

Running `./tests/run_examples.sh` on my network often fails at the `Selector with a wait` test. This happens because sometimes two seconds isn't enough for the selected element to load. ## Steps...

nielthiart

There's a neat little package [autoscraper](https://github.com/alirezamika/autoscraper) that allows to quickly build no-code web extractors. - You take a page with known content. - Say what text from it you need...

walking-octopus

I wonder if adding some mechanisms for keeping a session alive and using the CLI to send commands to it can help automate simple actions or hack together entire agents....

walking-octopus

Ability to pass CLI options to `shot-scraper multi`

3

https://discord.com/channels/823971286308356157/1034259126076833873/1034303037914746942 @jefftriplett > I was curious about the `multi` subcommand and if it made sense to expose screenshot api options (width + height + quality) as cli options. it's not...

simonw

Issue running on M1 mac using conda

1

Traceback (most recent call last): File "/Users/axeljeremy/opt/miniconda3/bin/shot-scraper", line 5, in from shot_scraper.cli import cli File "/Users/axeljeremy/opt/miniconda3/lib/python3.9/site-packages/shot_scraper/cli.py", line 6, in from playwright.sync_api import sync_playwright, Error, TimeoutError File "/Users/axeljeremy/opt/miniconda3/lib/python3.9/site-packages/playwright/sync_api/__init__.py", line 25, in...

axeljeremy7

bash interpolation inside shots.yml

1

Hi there! thanks so much for this awesome library. I've been using it to archive photos of my website every day to keep a history. One thing I'm having trouble...

spencerc99

Add --proxy parameter to CLI interface

4

Please add `--proxy` parameter to CLI interface (https://playwright.dev/python/docs/network#http-proxy): cli.py ```python def proxy_option(fn): click.option("--proxy", help="HTTP Proxy to use")(fn) return fn # other options @proxy_option def shot(...): ... def _browser_context( p, auth,...

kirillklimenko

Proposal: add option to combine actions

1

I believe it would be handy to both being able to screenshot a page and scrape some other data (e.g. title etc.) at the same time. Right now it is...

nekromoff