crawl4ai icon indicating copy to clipboard operation
crawl4ai copied to clipboard

[Bug]: CRAWL4AI_API_TOKEN env var not enforced in Docker image

Open PinkFrog opened this issue 8 months ago • 7 comments

crawl4ai version

docker.io/unclecode/crawl4ai:latest

Expected Behavior

Title: CRAWL4AI_API_TOKEN env var not enforced in Docker image

Description:
When deploying docker.io/unclecode/crawl4ai:latest to Google Cloud Run with the CRAWL4AI_API_TOKEN environment variable (either directly or via Secret), the API still allows crawl requests without providing that token. The application appears not to be reading or enforcing the token at all.


Environment:

  • Image: docker.io/unclecode/crawl4ai:latest
  • Deployment: Google Cloud Run (env var set via UI)
  • No errors or log messages indicating missing token

Additional context:

  • Passing the token as a Secret or plain env var makes no difference.
  • No log entries show reading or validation of the env var.

Please let me know if you need any more details!

Current Behavior

Expected behavior:
Requests without a valid CRAWL4AI_API_TOKEN should be rejected (e.g. HTTP 401 Unauthorized).

Actual behavior:
All requests are accepted, regardless of whether CRAWL4AI_API_TOKEN is present or correct.

Is this reproducible?

Yes

Inputs Causing the Bug


Steps to Reproduce

**Steps to reproduce:**
1. Deploy the image to Cloud Run using:
   - Image: `docker.io/unclecode/crawl4ai:latest`
   - Container port
   - Environment variable `CRAWL4AI_API_TOKEN` set to any value  
2. Send an HTTP request to any crawl endpoint **without** including the API token.  
3. Observe that the request succeeds and crawl proceeds.

Code snippets


OS

Google Cloud Run

Python version

docker

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

No response

PinkFrog avatar Apr 22 '25 23:04 PinkFrog

can confirm, have it selfhosted with dockerimage:latest on Coolify and set CRAWL4AI_API_TOKEN in ENV Vars. Did work with previous release.

has been removed from docker-compose.yml in the latest release. maybe this is the reason?

  • CRAWL4AI_API_TOKEN=${CRAWL4AI_API_TOKEN:-}

julianlangweb avatar Apr 23 '25 09:04 julianlangweb

any news about this?

julianlangweb avatar Apr 25 '25 08:04 julianlangweb

I encountered this issue as well. In the previous version, the CRAWL4AI_API_TOKEN in my ENV was checked correctly.

In the latest version, the TOKEN is not being checked, and I can execute the API without providing a TOKEN.

PhantasWeng avatar Apr 25 '25 12:04 PhantasWeng

statement from unclecode in discord:

"Yeah, I removed it. I am moving toward a new design that lets you run about 100 of these containers, each one as a browser. Then we will have a service layer connected to a load balancer, where we manage API token stuff."

just FYI all. maybe we get authentication with next releases i guess

julianlangweb avatar Apr 28 '25 18:04 julianlangweb

another update from unclecode via discord:

Oh I see, then this one still is there, I misunderstood. In config.yaml, under security, set enabled and jwt_enabled both to True, then check https://github.com/unclecode/crawl4ai/blob/main/tests/docker/test_server_token.py to see how you can generate JWT token, and use it. If you want to use in production, you have to make a few changes. It designed to be used for multiple users, or just a single user.

julianlangweb avatar May 02 '25 09:05 julianlangweb

CRAWL4AI_API_TOKEN is deprecated now! Now there's no need for any API token, but if you want to use in production with jwt token enable security in settings.

aravindkarnam avatar May 14 '25 12:05 aravindkarnam

how to get the api_token in dify?

minglong-huang avatar May 22 '25 15:05 minglong-huang

To be fixed in: https://github.com/unclecode/crawl4ai/blob/main/docs/examples/docker_example.py https://github.com/unclecode/crawl4ai/blob/main/tests/docker_example.py

aravindkarnam avatar Aug 05 '25 10:08 aravindkarnam

这是来自QQ邮箱的假期自动回复邮件。   您好,我最近正在休假中,无法亲自回复您的邮件。我将在假期结束后,尽快给您回复。

minglong-huang avatar Aug 05 '25 10:08 minglong-huang

@aravindkarnam

CRAWL4AI_API_TOKEN is deprecated now!

Does this mean there's no bearer token security option for REST API calls anymore?

mk0y avatar Aug 06 '25 11:08 mk0y

@mk0y the static CRAWL4AI_API_TOKEN key was dropped, but bearer token auth is still there, it just moved to a JWT-based flow.

SohamKukreti avatar Aug 07 '25 06:08 SohamKukreti

already merged with the main branch and the latest version (0.7.4)

ntohidi avatar Aug 18 '25 03:08 ntohidi