scrapy-playwright
scrapy-playwright copied to clipboard
scrapy-plugins
🎭 Playwright integration for Scrapy
Hello, I have install scrapy-playwright in my venv using `pip install scrapy-playwright` and after that `playwright install`. Whatever I try to do I get the ERR_INVALID_ARGUMENT error for any url....
Hi all, I have a simple example below which should work but doesn't. ```python class AwesomeSpider(scrapy.Spider): name = "test-playwright" def start_requests(self): yield scrapy.Request("https://quotes.toscrape.com/", meta={ "playwright": True, }) def parse(self, response):...
These is pure playwright code, and it can login ti well. ```python from playwright_stealth import StealthConfig from playwright_stealth import stealth_async async def playwright_ti_jiaocha(): from playwright.async_api import async_playwright async with async_playwright()...
I'm working on a project with a large number of spiders and a fairly sizable community of open contributors. To ensure that things are working correctly over the long-term, I...
I am using scrapy-playwright with brave. I find out that the code with add "pragma" and "cache-control" as an header. Is it possible to remove it? Usually, we should use...
I have tested it on win11, please check it, hope it helpful. scrapy_playwright/handler.py : ```python import asyncio import logging from contextlib import suppress from dataclasses import dataclass from ipaddress import...
Scarpy is hanging when an exception is raised in applying page methods. Though scrapy is not reporting any exception when applying page method but I have verified by writing code...
Hello, I am trying to include scrapy-playwright to my binary using PyInstaller. I have tried a few different setups: - When installing scrapy-playwright: ``` python3 -m venv .venv source .venv\bin\activate...
Requested in #20 Introduce a new `PLAYWRIGHT_CLOSE_CONTEXT_INTERVAL: int` setting. Contexts that have spent the specified amount of seconds without pages will be automatically closed.
