crawlee-python icon indicating copy to clipboard operation
crawlee-python copied to clipboard

The queue still contains requests locked by another client - false positive

Open Pijukatel opened this issue 8 months ago • 0 comments

RequestQueue can log following line even in situation where there is only one client and thus no other client exists: The queue still contains requests locked by another client

For example when queuing the very first request in freshly started crawler:

2025-04-02T09:15:44.576Z Actor is running on the Apify platform, `disable_browser_sandbox` was changed to True.
2025-04-02T09:15:44.787Z [apify] INFO  Initializing Actor...
2025-04-02T09:15:44.790Z [apify] INFO  System info ({"apify_sdk_version": "2.5.0", "apify_client_version": "1.9.2", "crawlee_version": "0.6.5", "python_version": "3.13.2", "os": "linux"})
2025-04-02T09:15:44.962Z [BeautifulSoupCrawler] INFO  Current request statistics:
2025-04-02T09:15:44.964Z ┌───────────────────────────────┬──────────┐
2025-04-02T09:15:44.971Z │ requests_finished             │ 0        │
2025-04-02T09:15:44.974Z │ requests_failed               │ 0        │
2025-04-02T09:15:44.975Z │ retry_histogram               │ [0]      │
2025-04-02T09:15:44.977Z │ request_avg_failed_duration   │ None     │
2025-04-02T09:15:44.978Z │ request_avg_finished_duration │ None     │
2025-04-02T09:15:44.980Z │ requests_finished_per_minute  │ 0        │
2025-04-02T09:15:44.981Z │ requests_failed_per_minute    │ 0        │
2025-04-02T09:15:44.983Z │ request_total_duration        │ 0.0      │
2025-04-02T09:15:44.984Z │ requests_total                │ 0        │
2025-04-02T09:15:44.986Z │ crawler_runtime               │ 0.035429 │
2025-04-02T09:15:44.987Z └───────────────────────────────┴──────────┘
2025-04-02T09:15:44.989Z [crawlee._autoscaling.autoscaled_pool] INFO  current_concurrency = 0; desired_concurrency = 2; cpu = 0; mem = 0; event_loop = 0.0; client_info = 0.0
2025-04-02T09:15:45.188Z [crawlee.storages._request_queue] INFO  The queue still contains requests locked by another client

Pijukatel avatar Apr 02 '25 11:04 Pijukatel