crawl4ai icon indicating copy to clipboard operation
crawl4ai copied to clipboard

[Bug]: Webhook API within Docker missing CrawlerConfig/Browser config

Open devonik opened this issue 3 weeks ago • 1 comments

crawl4ai version

latest

Expected Behavior

I successfully build a crawler request with BrowserConfig and CrawlerConfig with CSSExtraction, etc. Now I want to build a webhook strategy to not wait for the crawler response so I used this from the documentation https://docs.crawl4ai.com/core/docker-deployment/#job-queue-webhook-api

Current Behavior

The request body of /crawl/job does not accept BrowserConfig and CrawlerConfig as /crawl does. How can I define them?

Is this reproducible?

Yes

Inputs Causing the Bug


const crawler_config_payload: CrawlerRunConfig = {
    type: 'CrawlerRunConfig',
    params: {...}
}

const crawl_payload = {
      urls: searchURLs,
      browser_config: browser_config_payload,
      crawler_config: crawler_config_payload,
    }
const response: {
      results: {
        extracted_content: string
      }[]
    } = await $fetch(`${config.crawl4AiUrl}/crawl/job`, {
      method: 'post',
      body: crawl_payload,
    })

Steps to Reproduce


Code snippets


OS

macOS

Python version

DOcker

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

No response

devonik avatar Nov 14 '25 22:11 devonik

Any answer ? I want to introduce webhooks cause my request to the docker API is getting a timeout after, 60000. I tried to increase the timeout in my Axios client and the timeout of crawl4ai, but it does not work. I'm also open to other approaches to prevent the timeout.

devonik avatar Nov 26 '25 16:11 devonik