feat: adding crawlee's EnqueueStrategy config

[muzafferkadir]

PR Description

Summary: This pull request introduces changes to the configuration schema and the crawling logic to enhance the flexibility of the crawling strategy. For more information: https://crawlee.dev/api/core/enum/EnqueueStrategy#All Changes Made:

1. Updated Configuration Schema (config.ts):

   • Added crawlStrategy field to the configuration schema.
     • This field allows specifying the Crawlee strategy for checking certain parts of the URLs found.
     • Possible values are "all", "same-origin", "same-hostname", and "same-domain".
     • This field is optional.

2. Updated Crawling Logic (core.ts):

   • Integrated the crawlStrategy configuration into the PlaywrightCrawler setup.
     • The strategy parameter in enqueueLinks now uses the config.crawlStrategy value if provided.
     • Ensures that the crawling strategy defined in the configuration is applied during the crawling process.

Impact:

• These changes provide greater control over the crawling behavior, allowing users to specify how URLs are handled based on their origin and domain.

Examples:

• When crawlStrategy is set to "same-origin", the crawler will only follow links within the same origin.
• When crawlStrategy is set to "all", the crawler will follow all links regardless of their origin.

[steve8708]

thanks @muzafferkadir ! looks like build is failing, so will need that in to merge. otherwise, great update

[muzafferkadir]

thanks @muzafferkadir ! looks like build is failing, so will need that in to merge. otherwise, great update

thanks, i updated

[steve8708]

sorry @muzafferkadir - looks like theres a merge conflict. i can hop on this once green again

[muzafferkadir]

sorry @muzafferkadir - looks like theres a merge conflict. i can hop on this once green again

i updated

[steve8708]

looks like build not passing @muzafferkadir