crawl4ai
crawl4ai copied to clipboard
[MOD]CrawlerRunConfig json to object, (import crawl4ai)need import mo…
…re Object
Summary
Fixes #1056
List of files changed and why
crawl4ai/init.py
How Has This Been Tested?
"crawler_config": {
"type": "CrawlerRunConfig",
"params": {
"chunking_strategy": {
"type": "SlidingWindowChunking",
"params": {}
},
}
}
crawler_config json can deserialize to CrawlerRunConfig
Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] I have added/updated unit tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
Summary by CodeRabbit
-
New Features
- Introduced several new chunking strategies, offering more options for processing and segmenting content.
- Added a new naive JSON LXML extraction strategy for expanded extraction capabilities.
-
Refactor
- Updated and reorganized the list of available extraction and chunking strategies for improved clarity and usability.