crawlee-python icon indicating copy to clipboard operation
crawlee-python copied to clipboard

Feature Request: Add More Options to Crawlee CLI for Crawler Creation

Open siddiqkaithodu opened this issue 6 months ago • 0 comments

At the moment, the Crawlee CLI provides two options for creating a crawler in the CLI:

  1. Beautiful Soup
  2. Playwright

It would be great if we could add more alternatives, such as selecting the request module (like curl cffi,httpx) and parser modules (like parsel). This would allow the project to be bootstrapped with the chosen dependencies.

Additionally, it would be super helpful to include the ability to specify the domain name in the CLI we want to start with, similar to how it’s done in Scrapy.

siddiqkaithodu avatar Aug 09 '24 11:08 siddiqkaithodu