feat: bacth operation should be retried if some of them are failed

[Xuanwo]

https://github.com/datafuselabs/opendal/blob/0eff4b7ea52147b40b0406141516fddb09e9d0f5/src/layers/retry.rs#L287-L300

We only retried batch request it self and didn't retry if some of them are failed. Maybe we should also retry them.

[suyanhanx]

Add a parameter to indicate whether internal requests are required to retry or not? After all, there are some batches that must succeed completely.

Their idempotency may need to be determined by the application itself?

[Xuanwo]

At the moment, we only have the capability to delete in batches. Therefore, I believe it would be acceptable to attempt the process again.

[nasen23]

I could attempt to implement this, but what should the retry strategy be?

Here's a suggested strategy: After executing a batch operation, if some of the operations fail, we would retry the failed operations as a batch. Continue retrying until the maximum number of retries is reached.

[Xuanwo]

Retry the whole batch is good enough.

[suyanhanx]

Retry the while batch is good enough.

Did you mean 'whole'?

[Xuanwo]

Did you mean 'whole'?

Aha, yes. typo fixed.

[nasen23]

Retry the whole batch is good enough.

That's a lot easier to implement 😁. Please assign this to me.

[Xuanwo]

That's a lot easier to implement . Please assign this to me.

Excellent! I have assigned this task to you. Enjoy!