botasaurus
botasaurus copied to clipboard
Is it possible to integrate custom scraping task in a class?
By integrating custom scraping task _request_scrape_data into a class in this way
class RequestCrawlasaurus():
def __init__(self, start_url=""):
self.start_url= start_url
@request(
output=None,
create_error_logs=False,
raise_exception=True,
parallel=3,
)
def _request_scrape_data(self, request: AntiDetectDriver, link):
response = request.get(link)
def crawl(self):
urls = self._request_scrape_data(link=self.start_url)
I got this error: "RequestCrawlasaurus._request_scrape_data() missing 1 required positional argument: 'link'"
Is it possible to embed the custom scraping task _request scrape data into a class?
If yes, how?