Scrapegraph-ai icon indicating copy to clipboard operation
Scrapegraph-ai copied to clipboard

Burr Integration throws errors with SmartScraperGraph example

Open flst01 opened this issue 6 months ago • 1 comments

Describe the bug When using the example (https://scrapegraph-ai.readthedocs.io/en/latest/getting_started/examples.html) with Burr integration (https://scrapegraph-ai.readthedocs.io/en/latest/scrapers/graph_config.html#burr-integration) Fetch throws an error. After fixing the error in Fetch, ParseNode throws an error.

To Reproduce Steps to reproduce the behavior: After starting burr in terminal by using "burr", run the following: "import os from dotenv import load_dotenv from scrapegraphai.graphs import SmartScraperGraph from scrapegraphai.utils import prettify_exec_info

load_dotenv() openai_key = os.getenv("OPENAI_API_KEY")

graph_config = { "llm": { "api_key": openai_key, "model": "openai/gpt-4.1-nano", }, "burr_kwargs": { "project_name": "test-scraper", "app_instance_id": 1, } }

smart_scraper_graph = SmartScraperGraph( prompt="List me all the projects with their description.", source="https://perinim.github.io/projects/", config=graph_config )

result = smart_scraper_graph.run()"

Expected behavior Burr integration should work without error

Acutal Behaviour Error in Fetch: ValueError: Action ParseNode attempted to write to keys {'content'} that it did not declare. It declared: (['parsed_doc'])! Error in ParseNode: ValueError: Action ParseNode attempted to write to keys {'content'} that it did not declare. It declared: (['parsed_doc'])!

Desktop :

  • OS: macOS Sequoia 15.4.1

Additional context Solved by fork. See Pull Request. Might still affect other Graph types, havent checked...

flst01 avatar May 08 '25 01:05 flst01