Improve docs about request locking

[drobnikj]

Which package is the feature request for? If unsure which one to select, leave blank

@crawlee/core

Feature

After releasing the new request queue storage system, we should improve the docs about locking. Currently, there is outdated information in experiments, see https://crawlee.dev/docs/next/experiments/experiments-request-locking .

We can add info and explanation about locking somewhere into crawlee docs.

Motivation

Inform users of crawlee how the locking works under the hood.

Ideal solution or implementation, and any additional constraints

The ideal state would be to have documented how locking works under the hood in crawlee.

Alternative solutions or implementations

No response

Other context

No response

[B4nan]

Is this for your team or do you want us to handle it?

[B4nan]

@drobnikj let me ask again, is this for you or for us? :]

[drobnikj]

@B4nan I think It would be good task for @vladfrangu as he did the implementation in crawlee. I think we need to add:

• info on how locking works in crawlee
• example of how you can use locking to scrape one request queue from multiple crawlee instances
• remove the section about experiments request queue locking