Scrapegraph-ai
Scrapegraph-ai copied to clipboard
ScrapeGraphAI
Python scraper based on AI
I want to add this crawler to [langflow ](https://github.com/langflow-ai/langflow)，Can you tell me the necessary feasibility.
I am developing a product that requires converting any webpage into an RSS feed (in XML or JSON format). If an RSS feed URL is already available (thus no need...
Even though I applied the filtering parameter, SearchLinkGraph continues to give links with extensions such as .css and .js. What should I do? graph_config = { "llm": { "model": "ollama/llama3.2",...
**Is your feature request related to a problem? Please describe.** When trying to automate the process of scraping i've found that trying to run the `local` browser inside a lambda...
**Describe the bug** When using the example (https://scrapegraph-ai.readthedocs.io/en/latest/getting_started/examples.html) with Burr integration (https://scrapegraph-ai.readthedocs.io/en/latest/scrapers/graph_config.html#burr-integration) Fetch throws an error. After fixing the error in Fetch, ParseNode throws an error. **To Reproduce** Steps to...
the current version does not have the ability to scrape such sites with dynamically loaded content https://www.freeletics.com/ https://sider.ai/en
**Describe the bug** Burr package is required in order to start any example of ScrapeGraphAI. **To Reproduce** ```bash pip install scrapegraphai playwright install python3 smart_scraper_openai.py # examples/openai/smart_scraper_openai.py ``` **Expected behaviour**...
**Is your feature request related to a problem? Please describe.** Implement a local HTML caching. To avoid re-process all scraping each time. It's notably long with recursive depths scrapings. **Describe...
